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Summary. Let Y(gl n ) be the Yangian of the general linear Lie algebra gl n . Let Y(sp n ) and 
Y(so n ) be the twisted Yangians corresponding to the symplectic and orthogonal subalgebras 
■ in the Lie algebra Ql n . These twisted Yangians are one-sided coideal subalgebras in the Hopf 

algebra Y(gi n ) . We give realizations of irreducible modules of the algebras Y(sp n ) and Y(so n ) , 
as certain quotients of tensor products of symmetic and exterior powers of the vector space 
C n . For the Yangian Y(gl n ) such realizations have been known, but we give new proofs of 
^-j- [ these results. For the twisted Yangian Y(sp n ), we realize all irreducible finite-dimensional 

£> ' modules. For the twisted Yangian Y(so n ), we realize all those irreducible finite-dimensional 

modules, where the action of the Lie algebra so n integrates to an action of the complex special 
orthogonal Lie group SO n . Our results are based on the theory of reductive dual pairs due to 
Howe, and on the representation theory of Mickelsson algebras. 
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Mickelsson algebras are natural objects used for the study of Harish-Chandra modules 
of real reductive Lie groups. They appeared in [M] first. Their detailed study was later 
undertaken by Zhelobenko [Zl, Z2] who employed the theory of extremal projectors for 
reductive Lie algebras due to Asherova, Smirnov and Tolstoy [AST]. The Mickelsson 
algebras were further studied by Ogievetsky and the first named author of this article 
[K, KO]. In particular, these works dealt with the extremal cocycle on the Weyl group 
of any reductive Lie algebra, introduced in [Zl]. 

In [Zl] a Mickelsson algebra was defined by a finite-dimensional complex Lie algebra 
a and its reductive subalgebra g. Following [K, KO] we define Mickelsson algebras in a 
more general setting. Let us take any associative algebra A over the complex field C, 
containing the universal enveloping algebra U(g) as a subalgebra. Suppose there is a 
vector subspace V C A, invariant and locally finite under the adjoint action of g, such 
that the multiplication map U(g) <S> V — > A is bijective. Choose a Borel subalgebra of g 
with the Cartan subalgebra t and the nilpotent radical n. Take the right ideal J = nA 
of the algebra A and consider the normalizer Norm J C A of this ideal. Our Mickelsson 
algebra R is the quotient of the algebra Norm J by its two-sided ideal J. In the case 
when A = U(o) , our R becomes the Mickelsson algebra considered in [Zl]. 
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Consider the ring of fractions A of the algebra A relative to the set of denominators 
(1.3). Take the right ideal J = nA of the algebra A and its normalizer NormJ C A. 
Then define the algebra R as the quotient of NormJ by J. Unlike R, the algebra R 
admits a rather nice description. Let n' be the nilpotent subalgebra of g opposite to 
n. Let J' = An' and J' = An' be the corresponding left ideals of A and A. Consider 
the quotient vector space Z = A/(J + J'). Taking the elements of R modulo J' defines 
a map R — > Z. This map is bijective. The multiplication in R can be described in 
terms of the vector space Z, by using the extremal projector for g; see Subsection 1.2. 
Furthermore, according to [KO] the extremal cocycle corresponding to the Weyl group 
of g determines an action of the braid group of g by automorphisms of the algebra Z . We 
call them the Zhelobenko automorphisms of Z . This braid group action is closely related 
to the dynamical Weyl group action of Etingof, Tarasov and Varchenko [EV, TV1, TV2]. 

The adjoint action of the Lie algebra g on A determines an action of the Cartan 
subalgebra t C g on Z. Consider the subalgebra of Z consisting of all elements of zero 
weight relative to t. The Zhelobenko automorphisms of Z preserve this subalgebra, and 
moreover determine on it an action of the Weyl group of g, see Subection 1.4 below. 

Let G be a reductive algebraic group over C with the Lie algebra g. Suppose there 
is an action of the group G by automorphisms on the algebra A, extending the adjoint 
action of G on the subalgebra U(g) c A. Suppose the action of G on A preseves the 
subspace V, and that the action of G on V is locally finite. Suppose the action of g on 
A corresponding to that of G coincides with the adjoint action. Our joint work with 
Vinberg [KNV] describes the image Q of the subalgebra of G -invariant elements A G C A 
under the projection A — y A/(J + J'). It shows that Q consists of all elements which 
have weight zero relative to t, are invariant under the Zhelobenko automorphisms of 
A/( J + J') , and are also invariant under certain other automorphisms arising when the 
group G is not connected; see Subsection 1.5. Here A/(J + J') is regarded as a subspace 
of Z = A/( J + J') . The Zhelobenko automorphisms do not preserve this subspace in 
general. Thus using the ring of fractions A is necessary for this description of A G . 

The present work is a continuation of [KNV] and also of a series of our publications 
[KN1, KN2, KN3, KN4]. The latter series has established correspondences between the 
Zhelobenko automorphisms of certain algebras of the form Z discussed above, and the 
canonical interwining operators of tensor products of representations of Yangians. These 
are the Yangian Y(gl n ) of the general linear Lie algebra gl n , and its twisted analogues 
Y(sp n ) and Y(so n ) corresponding to the symplectic and orthogonal subalgebras sp n and 
so n of gi n . For an introduction to these Yangians see the recent book [M2]. The Yangian 
Y(gl n ) is a Hopf algebra while the twisted Yangians Y(sp n ) and Y(so n ) are one-sided 
coideal subalgebras of Y(gl n ) . These Yangians admit homomorphisms to the rings of 
the form A G discussed above. Moreover, together with the subalgebra U(g) G C A G , the 
image of each of these homomorphisms generates the corresponding ring of invariants 
A . These homomorphisms first arose in the works of Olshanski [01, 02] and are also 
discussed in our Subsection 4.3. Another connection between the works [01, 02] and the 
theory of Mickelsson algebras was used by Molev to construct weight bases of irreducible 
finite-dimensional modules of the Lie algebras sp n and so n ; see for instance [M2]. 

In the present work we combine the results of [KN1, KN2, KN3, KN4, KNV] to give 
realizations of irreducible representations of the twisted Yangians. In particular, every 
irreducible finite-dimensional module of the twisted Yangian Y(sp n ) will be realized, 
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up to changing the action of the centre of this algebra, as the image of an intertwining 
operator between representations of Y(sp n ) in tensor products of exterior powers of C n . 
For the Yangian Y(gl n ) , such realizations have been provided by the works of Akasaka 
and Kashiwara [AK] and of Cherednik [C2] . Our work gives new proofs of these results 
for Y(g[ n ) . For the twisted Yangian Y(so n ) , the images of our intertwining operators 
realize, up to changing the action of the centre of Y(so n ) , all those irreducible finite- 
dimensional modules, where the action of the subalgebra U(so n ) C Y(so n ) integrates to 
an action of the complex special orthogonal Lie group SO n . 

Now let U(g) C A be an arbitrary pair as described above. Let A be the zero weight 
component of the algebra A relative to the adjoint action of t. We denote by S the 
subalgebra in Z generated by the image of A C A under the projection A — > Z . Then 
Q C S by definition. For any weight A G t* let N be an irreducible S -module whereon 
the subalgebra U(t) C S acts via the mapping A : t — > C, extended to a homomorphism 
U(t) — > C. Then our Proposition 1.6. gives sufficient conditions for irreducibility of the 
restriction of N to the subalgebra Q C S. One of the conditions is that the weight A + p 
is nonsingular. Here p denotes the half-sum of positive roots, and nonsingularity means 
that the value of A + p on any positive coroot vector of t is not a negative integer. 

Our results on irreducibility of representations of the Yangians Y(g( n ) , Y(sp n ) and 
Y(so n ) are based on Proposition 1.6. By using the homomorphism A G — > Q determined 
by the projection A — > A/(J + J'), any irreducible Q-module becomes an irreducible 
module over the algebra A . By using the homomorphisms from the Yangians to the 
algebras of the form A G , we then obtain irreducible representations of these Yangians. 

Proposition 1.6 is the main result of Section 1. In that section we also collect several 
other results on the algebra Z corresponding to an arbitrary pair U(g) c A. Below we 
describe the contents of all other sections of our article in more detail. 

0.2. Overview of Section 2 

In this article, we employ the theory of reductive dual pairs due to Howe [HI] . Namely, 
we use the pairs (G,G') of complex reductive algebraic groups from the following list: 

(GL m ,GL n ), (0 2m ,SpJ, (Sp 2m ,O n ), (Sp 2m ,Sp n ), (0 2m ,O n ). (0.1) 

It is the Lie algebra g' of the group G' dual to G that will determine our Yangian Y(g') . 
We will employ both the symmetric and skew-symmetric versions of the Howe duality. 
Let U be the tensor product C m <g> C n of vector spaces. The group G' = GL n , Sp n , O n 
acts on U naturally, via its defining action on C n . In the symmetric case, this action of G' 
is extended from U to the Weyl algebra of U. The subalgebra of G'-invariant elements 
in the Weyl algebra then becomes a homomorphic image of the universal enveloping 
algebra U(g) where g is the Lie algebra of G = GL m ,0 2m ,Sp 2m respectively. In the 
skew-symmetric case, the action of G' on U is extended from U to the Clifford algebra 
of U. Then the subalgebra of G'-invariant elements in the Clifford algebra becomes 
a homomorphic image of U(g) where g is the Lie algebra of G = GL m ,Sp 2m ,0 2m 
respectively. Thus the first pair in the list (0.1) appears in both the symmetric and 
skew-symmetric cases, the next two pairs appear only in the symmetric case, while the 
last two pairs appear only in the skew-symmetric case. 
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We will denote by H (U) either the Weyl or the Clifford algebra, and will distinguish 
the symmetric and the skew-symmetric cases by using a parameter 9 which equals 1 or 
— 1 respectively. The homomorphism U(g) — > H(E/) will be denoted by (. Our algebra 
A will be the tensor product U(g) ®H(t/) . The algebra U(a) will be embedded into this 
tensor product diagonally: any element X G g will be identified with the element (2.6) of 
A . The subspace V C A will be then 1 <g> H (U) . The group G acts by automorphisms of 
the algebra H (U) , and the corresponding action of g on H (U) coincides with the adjoint 
to (. Thus we get a (diagonal) action of the group G by automorphisms of the algebra 
A, obeying the assumptions from the beginning of this section. In Subsection 2.1 we 
summarize the properties of the homomorphism £ : U(g) — > H(£7) that we will use. 

In Subsection 2.2 we introduce our main tool, the Shapovalov form on the algebra Z 
corresponding to A = U(g) ® H(£7) . It takes values in the ring of fractions U(t) of the 
algebra U(t) relative to the set of denominators (1.3). The algebra H(£7) is generated by 
its two subspaces, U and the dual U* . Let P (U) be the subalgebra of H (U) generated by 
the subspace U. Then P (U) is the symmetric algebra of U when 9 = 1, or the exterior 
algebra of U when 9 = — 1 . For any fiGt* let be the corresponding Verma module 
of the Lie algebra g. There is a unique weight k G t* satisfying the condition (2.12). 
This weight is zero if g — gl m , but differs from zero if g = sp 2m or g = S02 m ■ 

In Subsection 2.3 we take the space of n-coinvariants of the g-module <g> P(U). 
This space is denoted by M M . It is identified with a quotient space of the algebra A by 
the right ideal J and by a certain left ideal depending on p,, see the definition (2.14). 
If the weight p + k is generic, that is if the value of p + k on any coroot vector of t is 
not an integer, then the U(t) -valued Shapovalov form on Z defines a C-valued bilinear 
form on M^. The latter form is denoted by S^. Proposition 2.3 relates the form to 
the extremal projector for the Lie algebra g . 

In Subsection 2.4 for any A G t* we take the subspace of weight A in the space M M . 
This subspace is denoted by . If ji + k is generic, then we define a C-valued bilinear 
form on the subspace C as the restriction of the form . Letting the weights 
A and fi vary while the difference A — fi is fixed, we extend the definition of to all \i 
such that A + p is nonsingular, by continuity. Thus here the weight fi + k needs not to 
be generic. This extension of corresponds to the fusion procedure of Cherednik [C2] , 
see below for an explanation of this correspondence. Our Proposition 2.6 states that the 
quotient of by the kernel of the bilinear form is an irreducible S -module. This is 
the main result of Section 2. It shows that under the extra conditions of Proposition 1.6 
the quotient of by the kernel of is an irreducible Q -module, see Corollary 2.7. 

In Subsection 2.5 we give another construction of irreducible Q-modules, by using 
the general results on the Mickelsson algebras from our Subsection 1.6. Let L M be the 
quotient of the Verma module M M by its maximal proper submodule . Take the space 
of n-coinvariants of weight A of the fj-module L M ®P (U) . We denote this space by L^ . If 
X + p is nonsingular, then L^ is an irreducible S -module. Moreover, then under the extra 
conditions of Proposition 1.6 the space L^ is an irreducible Q-module, see Corollary 2.8. 
Note that then L^ is an irreducible A -module by Proposition 1.5. Thus we extend the 
results of Harish-Chandra [H] and of Lepowski and McCollum [LM] . 

Now take the space of n-coinvariants of weight A of the 0-module ® P(U) . We 
denote this space by N^ . Then L^ can be regarded as the quotient of the space of 
by its subspace N^ . In the case when X + p is nonsingular while fi satisfies the conditions 
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of Proposition 2.9, the subspace C coincides with the kernel of the form 
Hence the quotient of by the kernel of coincides with in this case. For each 
pair (G,G') from the list (0.1), weights p satisfying the conditions of Proposition 2.9 
will be produced later on, in Section 4. 

0.3. Overview of Section 3 

In Section 3 we give another interpretation of the quotient space of by the kernel of 
Sj}, for any p and nonsingular A + p. The extremal cocycle corresponding to the Weyl 
group & of g determines not only an action of the braid group of g by automorphisms 
of the algebra Z , but also an action of the same braid group by linear operators on the 
vector space A/ J. We call them the Zhelobenko operators on A/ J, see Subection 1.4. 
Throughout this article the symbol o indicates the shifted action of the group 6 on f, 
see (1.2). Again suppose that the weight p + k is generic. Then the quotient space M M 
of A can be identified with the quotient space of A by the right ideal J and by a certain 
left ideal. In Subsection 3.1 we describe the image of the quotient M M of A under the 
action of the Zhelobenko operator £ CT on A/ J, corresponding to any element a e &. If 
= gl m then this image can be identified with M aQfl . But if g = £p 2m or g = so 2m then 
our description of the image is more involved, see Corollary 3.4. For g = gl m this result 
was obtained in [KN1, KN2]. For g = sp 2m or g = $02m it was obtained in [KN3, KN4]. 
In Subsection 3.1 we give a new proof of this result, uniform for all these g. 

For generic p + k the operator £ CT on A/ J maps the subspace C of weight A 
to the subspace of weight a o A in the image of M M . Our Proposition 3.5 relates this map 
to the extremal projector for g . Letting the weights A and p vary while their difference 
A — p is fixed, we extend the definition of this map to all p such that X + p is nonsingular, 
again by continuity; see Corollary 3.6. Thus the weight p + k needs not to be generic 
anymore. Yet if p + k is not generic then the operator £ CT on A/ J does not necessarily 
define any map on the whole space M M . In Subsection 3.3 we consider the operator £ CTo 
where oo is the longest element of &. Our Proposition 3.7 states that the kernel of the 
corresponding map of coincides with the kernel of the bilinear form , for any p 
and nonsingular X + p. The proof is based on Propositions 2.3 and 3.5. Corollary 2.7 
then implies that the quotient space of by the kernel of the map defined by £ ao is 
an irreducible Q -module, under the extra conditions of Proposition 1.6. Moreover, then 
by Proposition 1.5 this quotient space of is an irreducible A G -module. These are 
the main results of Section 3, they are stated as Theorem 3.8 and Corollary 3.9. 

0.4- Overview of Section 4 

In Section 4 we apply the results of Section 3 to the representation theory of the Yangians 
Y(g') where g' = gl n ,5p n ,so n according to (0.1). In Subsection 4.1 we fix realizations of 
the groups G and G' appearing in (0.1), and define the corresponding homomorphisms 
C : U(g) -» H(C7) . In Subsection 4.2 we recall the definitions of the Yangian Y(gl n ) and 
the twisted Yangians Y(sp n ), Y(so n ) . The latter two are defined as certain subalgebras 
of Y(gl n ) . Their definition implies that they are also right coideals of Y(gi n ) , see (4.22). 

The twisted Yangians Y(sp n ), Y(so n ) can also be defined in terms of generators and 
relations, see (4.14) and (4.18). The collection of all relations (4.14) can be written as 
the reflection equation, introduced by Cherednik [CI] and Sklyanin [S]. The algebras 
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denned by the relations (4.14) only, that is without imposing the relations (4.18), are 
called the extended twisted Yangians. They are denoted by X(sp n ), X(so n ) respectively. 
Thus for g' = sp n ,so n we have a surjective homomorphism X(g') — y Y(g'). Its kernel 
is generated by certain central elements of X(g') , see Subsection 4.2 for their definition. 

In Proposition 4.1 for A = U(g( m ) ®H([/) and 9 = 1, —1 we define a homomorphism 
Y(gl n ) — y A GLm . Its image and the subalgebra U(gl m ) GLr ™ ® 1 C A together generate 
A GLm . It is related to the homomorphism Y(gl n ) — y U(gl n+; ) GLi defined for / = 1, 2, . . . 
by Olshanski [01]. It is also related to the Cherednik functor studied by Arakawa and 
Suzuki [AS] and to the Drinf eld functor [Dl]. We explained the relations in [KN1, KN2]. 

Now consider any pair (G,G ; ) from the list (0.1) other than the pair (GL m ,GL n ). 
In Proposition 4.2 for A = U(g) §>> H(£7) we define a homomorphism X(g') — y A G . Its 
image together with the subalgebra U(g) ®lcA generate A . This homomorphism 
is related to the homomorphisms Y(sp n ) — y U(sp n+2Z ) Sp2i and Y(so n ) — y U(so n+ ;) 0i 
defined for any / = 1, 2, ... by Olshanski [02]. We explained the relation in [KN3, KN4]. 
To prove our Propositions 4.1 and 4.2 we use the classical invariant theory, like Molev 
and Olshanski [MO] did when studying the homomorphisms defined in [01, 02]. 

For any g' = gl n ,sp n ,so n the algebra Y(g') contains the universal enveloping algebra 
U(g') as a subalgebra, and admits a homomorphism Y(g') — y U(g') identical on that 
subalgebra. For g' = gi n this fact has been well known, see for instance the seminal work 
of Kulish, Reshetikhin and Sklyanin [KRS]. For g' = sp n ,so n this is another result from 
[02]. If 9 = 1 then denote by $> k the kth symmetric power of the defining gl n -module 
C n . If 9 = — 1 then denote by the same symbol <& k the kth exterior power of the g( n - 
module C n . Using the homomorphism Y(gl n ) — y U(gl n ), regard @ k as a module over 
the Yangian Y(gl n ) . For any t E C denote by @ k the Y(g( n ) -module obtained by pulling 
the Y(gl n ) -module <P k back through the automorphism (4.6) where z = 9t. 

The definition of the Hopf algebra Y(gl n ) employs a certain n x n matrix T(x) . The 
entries of this matrix are formal power series in x~ x where x is the spectral parameter. 
The coefficients of these series are generators of Y(gl n ). For G' = Sp n or G' = O n we 
denote by T(x) the transpose to the matrix T(x) relative to the bilinear form on C™ 
preserved by the subgroup G' C GL n . Then the assignment (4.12) defines an involutive 
automorphism of the algebra Y(gl n ) . For any non-negative integer k we denote by <3?~[ k 
the Y(g( n ) -module obtained by pulling the <l> k back through the automorphism (4.12). 

By definition, for g' = sp n ,so n the subalgebra Y(g') C Y(gl n ) is generated by the 
coefficients of the series in x which arise as the entries of the nxn matrix T(— x) T{x) . 
It turns out that if g' = sp n and 9 = 1 , or if g' = so n and 9 = — 1 , then the restriction 
of the above defined Y(gt n ) -module ^^~ k to the subalgebra Y(g') C Y(gl n ) coincides 
with the restriction of the Y(g[ n ) -module <& k . Both cases correspond to g = S02 m - For 
further explanation of this phenomenon, see the end of Subsection 4.4. 

For any pair (G , G') from the list (0.1) and for any weights A,/iGt* the subspace 
of the quotient M M of the algebra A is a module over the subalgebra A G by definition. 
For (G,G') = (GL m ,GL n ) by using the homomorphism Y(gl n ) — y A GLm we can regard 
as a module over the algebra Y(gt n ). If non-zero, this Y(gt n ) -module is equivalent 
to a tensor product of certain modules of the form $ k . This is stated as Proposition 4.3, 
see [KN1, KN2] for the proof. Here we use the comultiplication (4.10) on Y(gl n ). 

For any other pair (G,G') from (0.1), we can regard as a module of the algebra 
X(g') by using the homomorphism X(g') — y A G . If non-zero, this X(g')-module is also 
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equivalent to a tensor product of certain modules of the form ^ , pulled back through 
an automorphism (4.15) of X(g'). This is a particular case of our Proposition 4.5, see 
[KN3, KN4] for the proof of that proposition. Here we first take the tensor product of the 
Y(gl n ) -modules $ k , then restrict the tensor product to the subalgebra Y(g') C Y(gl n ) , 
and then use the homomorphism X(g') — > Y(g') mentioned above. Equivalently, we can 
use the right Y(gl n ) -co module structure on the algebra X(g'); see Subsection 4.2. 

Now suppose A + p is nonsingular, while p is arbitrary. Consider again the mapping 
of the weight subspace defined by the Zhelobenko operator t- ao on A/ J. In the case 
(G,G') = (GL m ,GL n ) by replacing by an equivalent Y(gl n ) -module we obtain an 
intertwining operator from a tensor product of Y(gl n ) -modules of the form . The 
target module of this operator can be identified with the tensor product of the same 
Y(gl n ) -modules <P k as for the source module, but taken in the reversed order; see (4.49). 
The quotient by the kernel, or equivalently the image of this intertwining operator is an 
irreducible Y(g[ n )-module due to Corollary 3.9 and Proposition 4.1. 

In the case (G,C) = (GL m ,GL n ) we can regard the quotient space of as an 
Y(gl n ) -module. The Yangian Y(gl n ) acts on and hence on via the homomorphism 
Y(gl n ) — > A GLm . If X+p is nonsingular, then the Y(gl n ) -module is irreducible for any 
p, see Subsection 4.6. Moreover, then the conditions of Proposition 2.9 are satisfied, if 
for each positive root a of g\ m the number z a defined just before stating Proposition 4.4, 
is not zero. Under these conditions, the image of our intertwining operator (4.49) is not 
zero, and is equivalent to as Y(gl n ) -module. Thus we extend the results of [AS]. 

Now take a pair (G,G') from the list (0.1) other than (GL m , GL n ) . By replacing 
by an equivalent X(g') -module we again obtain an intertwining operator from a certain 
tensor product of modules of the form <l> k . The target module of this operator can 
be identified with the tensor product of the modules <I?~[ k corresponding to the tensor 
factors <I?t of the source module, and taken in the same order; see (4.58). Here we need 
not to apply to the source and the target X(g') -modules the automorphisms (4.15), 
because by Proposition 4.5 these automorphisms are the same for the source and for 
the target. Hence we may also regard both the source and the target as Y(g') -modules. 
Note that if g = so 2 m, then instead of the first tensor factor <t>~[ k of the target we can 
use $ k as well; see (4.59). This is because for g = S02 m the restrictions of $ k and $^~ k 
to the subalgebra Y(g') C Y(gl n ) coincide, and because we regard this subalgebra as a 
right coideal to define the tensor products. 

If G = Sp 2m then by Corollary 3.9 and Proposition 4.2 the image of our intertwining 
operator (4.58) is an irreducible Y(g') -module for both 9 = 1 and 6 = — 1, that is for 
both g' = so n and g' = sp n . But if G = 02 m then our Corollary 3.9 and Proposition 4.2 
imply Y(g')-irreducibility of the image only under an extra condition, that the stabilizer 
of the weight A + p in the extended Weyl group of g = S02 m is contained in the proper 
Weyl group. In Subsection 4.5 we remove this extra condition, but only for 9 = — 1, 
that is only for g' = so n . Namely, for G = 02m and 6 = — 1 we prove that the image 
of our intertwining operator is irreducible under the joint action of the algebra Y(so n ) 
and the group O n . Moreover, if n is odd then the image is irreducible under the action 
of Y(so n ) alone. But if n is even then the image is either an irreducible Y(so n ) -module, 
or is a direct sum of two non-equivalent irreducible Y(so n ) -modules; see Corollary 4.9. 

Now let (G, G') again be any pair from the list (0.1) other than (GL m , GL n ) . Regard 
the quotient space of M^ as X(g') -module. The extended twisted Yangian X(g') acts 
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on and hence on L^ via the homomorphism X(fj') — > A . In Subsection 4.6 we derive 
the same irreducibilty properties of the X(cj') -module L^, as described above for the 
image of our operator (4.58). Suppose that A + p is nonsingular. If G = Sp 2m then 
L^ is an irreducible X(f/)-:module for both g' = so n and g' = sp n . If G = 02m then 
the X(g') -module is irreducible under an extra condition, that the stabilizer of the 
weight A + p in the extended Weyl group of g = so 2m is contained in the proper Weyl 
group. If G = 02 m and 9 = —1 then L^ is irreducible under the joint action of X(so n ) 
and O n . Moreover, if n is odd then L^ the image is irreducible under the action of the 
algebra X(so n ) alone. But if n is even then L^ is either an irreducible X(so n ) -module, 
or is a direct sum of two non-equivalent irreducible X(50 n ) -modules. 

Let (G,G') once again be any pair from (0.1) other than (GL m ,GL n ) . Let A + p be 
nonsingular. When 9 = — 1 or n = 1, the conditions of Proposition 2.9 are satisfied if 
for each positive root a of g the number z a defined just before stating Proposition 4.6, 
is not zero. When 9 = 1 and n > 1, the conditions of Proposition 2.9 are satisfied if z a 
is not zero for each compact positive root a of g . Under these conditions, the image of 
our intertwining operator (4.58) is not zero, and is equivalent to L^ as X(g') -module. 
Moreover, then the image is equivalent to L^ under the joint actions of X(g') and G'. 

In Subsection 4.7 we once again consider the Y(g') -intertwining operators (4.49) 
and (4.58), for g' = gl n and g' = sp n ,so n respectively. We show that the target Y(g')- 
modules of these operators are dual to the source modules. We also show how this duality 
arises from the theory of Mickelsson algebras, when the weight A + p is nonsingular. 

0.5. Overview of Section 5 

In Section 5 we use our intertwining operators (4.49) and (4.58) to give realizations of 
irreducible representations of the Yangian Y(gl n ) and of the twisted Yangians Y(sp n ), 
Y(so n ). We call two Y(gl n ) -modules similar if they differ only by an automorphism 
(4.7) of Y(gi n ) , where g(x) is any formal power series in x~ x with coefficients from C 
and leading term 1 . Up to equivalence and similarity, the irreducible finite-dimensional 
Y(gi n ) -modules were classified by Drinfeld [D2], who generalized the classification given 
for n = 2 by Tarasov [Tl, T2]. It was then proved by Akasaka and Kashiwara [AK] that 
any of these modules can be realized as a quotient of a tensor product of Y(gl n ) -modules 
of the form ^ with 9 = — 1 ; see also the work of Chari and Pressley [CP]. Further results 
were obtained by Chari [C] and by Brundan and Kleshchev [BK]. Note that the works 
[AK] and [C] deal with representations of quantum affine algebras. For a connection to 
the representation theory of Yangians see the work of Molev, Tolstoy and Zhang [MTZ] . 

In Subsection 5.1 we give new proofs of the results from [AK] for Y(gi n ) . Namely, we 
prove that up to equivalence and similarity, any irreducible finite-dimensional Y(gl n )- 
module arises as the quotient by the kernel of intertwining operator (4.49) for 9 = — 1, 
some m and certain weights \,p of gl m . Here the weight A + p is nonsingular, that is 
satisfies the conditions (5.1). The difference v = A— p satisfies the conditions (5.2), which 
come from Proposition 4.4. This proposition was obtained in [KN1, KN2] and gives an 
explicit formula for the image under the operator (4.49) of a certain distinguished vector, 
called highest. For 9 = — 1 the conditions (5.2) guarantee that the image is not zero; 
see Theorem 5.1. 
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Now consider the twisted Yangians Y(g') where q' = sp n ,so n . Any automorphism 
(4.7) of Y(gl n ) determines an automorphism of the subalgebra Y(g') C Y(gl n ). We 
call two Y(g') -modules similar if they differ only by such an automorphism. Up to 
equivalence and similarity, the irreducible finite-dimensional Y(g') -modules have been 
classified by Mo lev; see [M2] for an exposition of the classification. No explicit realization 
of these Y(g') -modules had been known so far in general. It is provided by our results. 
Thus we extend the works of Molev [Ml] and the second named author of this article [N] 
which give explicit realizations of irreducible Y(g') -modules from a particular class. 

In Subsection 5.3 we prove that up to equivalence and similarity, any irreducible 
finite-dimensional Y(sp n ) -module arises as the quotient by the kernel of the intertwining 
operator (4.58) for 9 = — 1, some m and certain weights A,/x of sp 2m . Here the weight 
A + p is nonsingular, that is satisfies the conditions (5. 9), (5. 10) and (5.11). The weight 
z/ = A — p — k oi sp 2m satisfies the conditions (5. 12), (5. 13) and (5.14); see Theorem 5.3. 

Recall that the twisted Yangian Y(so n ) contains U(so n ) as a subalgebra. We shall 
call a finite-dimensional module over the algebra Y(so n ) an (Y(so n ), O n )-module, if the 
group O n also acts on this module, and the corresponding action of the Lie algebra so n 
of O n on this module coincides with the action obtained by restricting the action of 
Y(so n ) to the subalgebra U(so n ) . If n is odd then any irreducible (Y(so n ), O n ) -module 
is also irreducible over Y(so n ) . But if n is even then any (Y(so n ), O n ) -module is either 
irreducible over Y(so n ) , or splits into a direct sum of two irreducible Y(so n ) -modules, 
not equivalent to each other. The irreducible Y(so n ) -modules occuring in this way for 
any n are all those whose restriction to the subalgebra U(so n ) C Y(so n ) integrates to a 
module of the special orthogonal group SO n C O n . 

In Subsections 5.4 and 5.5 we prove that up to equivalence and similarity, any finite- 
dimensional irreducible (Y(so n ), O n )-module arises as the quotient by the kernel of the 
intertwining operator (4.58) for 9 = — 1 , some m and certain weights A, \x of so 2m • Here 
X+p is nonsingular, that is satisfies the conditions (5. 16), (5. 17). The weight v = X—p — K 
of so 2m satisfies the conditions (5. 18), (5. 19); see Theorems 5.4 and 5.5. 

Here for both g' = sp n ,so n the weight k of g = sp 2m ,S02 m respectively is defined by 
(2.12). The above mentioned conditions on the weight v of g come from Proposition 4.6. 
This proposition was obtained in [KN3, KN4] and gives an explicit formula for the 
image under the operator (4.58) of a certain distinguished vector, again called highest. 
For 9 = — 1 our conditions on v guarantee that the image is not zero. The notion of a 
highest vector relative to Y(g') for g' = sp n ,so n is discussed in Subsection 5.2. 

1. Mickelsson algebras 

1.1. Generalities 

Let g be any reductive complex Lie algebra of semisimple rank r. Choose a triangular 
decomposition 

g = n©t©n / (1.1) 

where t is a Cartan subalgebra, while n and n' are the nilpotent radicals of two opposite 
Borel subalgebras of g containing t. Consider the root system of g in t*. The set of the 
positive roots of g will be denoted by A + . 
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Let a.\ , . . . , a r G A + be simple roots. For each c = 1 , . . . , r let H c = G t be the 
coroot corresponding to the simple root a c , Let E c G n' and F c G n be the root vectors 
corresponding to the roots a c and — a c . We suppose that [E C ,F C ] = H c . Let £ be a 
Chevalley anti-involution on g. This is an involutive anti-automorphism of g identical 
on t, such that for every c = 1 , . . . , r one has e(E c ) = e c F c and e(F c ) = e~ x E c for some 
non-zero e c G C. 

Let & be the Weyl group of the root system of g . Let a\ , . . . , a r G & be the reflections 
in t* corresponding to the simple roots a\ , . . . , a r . We also use the induced action of 
the Weyl group 6 on the vector space t. It is defined by setting X(a(H)) = a~ 1 (X)(H) 
for all a G &, H G t and A G t*. Let p G t* be the half-sum of the positive roots. Then 
the shifted action o of the group & on the vector space t* is defined by setting 



It induces an action o of & on the commutative algebra U(t) , by regarding the elements 
of this algebra as polynomial functions on t*. In particular, then (o~oH)(\) = H(a~ 1 o\) 
for H G t. For any left g-module K and any A G t* we will denote by K x the subspace 
of weight A in K; we have v G K x if and only if Hv = X(H) v for every H G t. 

Let A be any complex associative algebra, containing as a subalgebra the universal 
enveloping algebra U(g) . Suppose there is also a vector subspace V C A, invariant under 
the adjoint action of the Lie algebra g on A, such that: 

(a) the multiplication map U(g) ® V — >A: X®Y\-^-XYis bijective; 

(b) the multiplication map V ® U(g) ->A:F®Xi4 YX is bijective; 

(c) the space V is locally finite relative to the adjoint action of g. 

Since the subspace V C A is invariant under the adjoint action of g, the conditions (a) 
and (b) are equivalent. The condition (c) means that V can be decomposed into direct 
sum of irreducible finite-dimensional g-modules. Since the adjoint action of g on U(g) 
is locally finite, due to (c) the same is true for the adjoint action of g on A . 

Let J be the right ideal of the algebra A generated by the elements of the subalgebra 
n C g . Let Norm J C Abe the normalizer of this right ideal, so that Y G Norm J if and 
only if yj C J. Then J is a two-sided ideal of Norm J. The quotient algebra 



is called a Mickelsson algebra. By definition, the algebra R acts on the space K/nK of 
n-coinvariants of any left A-module K . This space itself will be denoted by K n . 

Recall that U(g) C A. Hence we have U(t) C Norm J. It follows that the Mickelsson 
algebra R also contains U(t) as subalgebra. Due to the condition (a) above, R is a torsion 
free module over U(t) relative to the left multiplication. The condition (b) implies that 
R is a torsion free module over U(t) relative to the right multiplication, 

For each root a G t* let H a = ct v G t be the corresponding coroot. The Weyl group 
6 acts on the vector space t so that a : H a h-> H a ^ for any a G & . Denote by U(t) the 
ring of fractions of the commutative algebra U(t) relative to the set of denominators 



a o A = cr(A + p) — p. 



(1.2) 



R = Norm J/ J 



{H ce + z\aeA + , zeZ}. 



(1.3) 



The elements of this ring can be regarded as rational functions on the vector space t* . 
The elements of U(t) C U(t) are then regarded as polynomial functions on t* . LetU(g) 
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be the ring of fractions of the algebra U(g) relative to the same set of denominators 
(1.3). This ring is well defined, because U(g) satisfies the Ore condition relative to (1.3). 

Now regard (1.3) as a subset of A using the embedding of t C g . Due to the conditions 
(a,b,c) the algebra A also satisfies the Ore condition relative to its subset (1.3). Let A 
be the ring of fractions of A relative to the same set of denominators (1.3). Then A is 
a free left and free right U(t) -module, which contains A as a subalgebra. 

Denote by J the right ideal of the algebra A generated by the elements of n. Consider 
the normalizer Norm J c A of this right ideal. This normalizer coincides with the ring 
of fractions of Norm J relative to the same set of denominators (1.3) as before. Now J 
is a two-sided ideal of Norm J . The quotient algebra 

R = Norm J/ J 

bears the same name of Mickelsson algebra, as the quotient algebra R does. Note that 
J fl A = J , because the subalgebra t C g normalizes the subalgebra n C g . Hence the 
natural embedding A — > A determines an embedding R — > R. 

1.2. Double coset algebra 

For each positive root a G A + let E a G n' and F a G n be the corresponding root 
vectors. In paricular, for any simple root a = a c we have E a = E c and F a = F c . For 
any weight A G t* consider the infinite sum 

oo 

P a [X] = l + J2(-l) S {s } -(H a + \(H ce ) + l)...(H a + \(H a )+s)y 1 F^E s a (1.4) 

8=1 

where the denominators do not belong to the set (1.3) in general. The sum belongs to 
certain algebra which contains U(g) and needs not to be defined here; see [Z2, Section 1]. 
Take a G & and any reduced decomposition a = ■ ■ ■ o~d 1 ■ Here / is the length of a. 
Any such a decomposition defines a normally ordered sequence of positive roots : 

Pi = a c i 1 , 02 = o"di («d 2 ) > •••> Pi = o-^o-d.2 ■ ■ ■o-d l _ 1 (ad l ) . 
The product 

P ff [A]=P A [A]...P j8l [A] 

in the above mentioned algebra does not depend on the choice of a decomposition; see 
[Z2, Section 2]. Let cr be the longest element of the Weyl group 6 . Put P [A] = P ao [A] . 

The element P = P [A] with A = p is called the extremal projector for the reductive 
Lie algebra q ; its definition is due to Asherova, Smirnov and Tolstoy [AST] . The element 
P can be presented as an infinite sum of elements of the algebra U(g) . Thus it belongs to 
a certain completion U(g) of the latter algebra; the completion needs not to be defined 
here. Below are the basic properties of the extremal projector: 

P 2 =P; (1.5) 
E a P = and PF a = for a e A + ; (1.6) 
Pel + nUfg) and ?Gl + uffl)n'. (1.7) 
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For the proofs of (1.5) and (1.6) see [AST] and [Z2, Section 3]. The two properties (1.7) 
follow directly from the definition (1.4), which also implies that s(P) = P . Moreover, 

oo oo 

PGl + ^n s Ufg) and P e 1 + ^ Ufa) (n') s . (1.8) 

8=1 3 = 1 

Now let J' be the left ideal of A generated by elements of n'. Similarly, let J' be the 
left ideal of A generated by elements of n'. First consider the double coset vector space 

Z = A/(J + J'), 

where the quotient is taken by a left and a right ideal. Since the Cartan subalgebra 
t C Q normalizes the subalgebras n and n', the left and right multiplications in A by 
the elements of the algebra U(t) make the vector space Z a U(t)-bimodule. Relative to 
the adjoint action of t on Z , we have a weight decomposition 

Z = © Z A . (1.9) 

Aet* 

Next consider the double coset vector space 

Z = A/(J + J'). 

This vector space is a bimodule over the algebra U(t). Since (J + J')nA = J + J', the 
natural embedding A — > A determines an embedding of U(t)-bimodules Z — > Z. Let us 
now equip the vector space Z with a binary operation o defined by setting 

XoY = XPY for I,FgZ. 

We first define the coset XPY G Z in the case when X ,Y G Z by using the relations 
(1.6). Although P is an infinite sum of elements of U(fj), the condition (c) implies that 
only finitely many summands of XPY are non-zero cosets in Z in this case. Here we use 
(1.8). This definition of XPY extends to any cosets X ,Y G Z by U(t) -linearity. Note 
that each summand of P commutes with every element of the subalgebra U(t) C U(fj) . 

So Z becomes an associative algebra over C . We call it the double coset algebra. The 
quotient space A/ J is a left Z -module relative to an action, denoted by o too, defined by 

XoZ = XPZ for X el and ZgA/J. (1.10) 

Recall that R C R C A / J . Now let n : A / J — > Z be the canonical projection, so that 

n(A + J) = A + J + J' for A e A. 

Proposition 1.1. (i) The restriction of n to R is a monomorphism of algebras. 
(ii) The restriction of it to R is an isomorphism of algebras R and Z. 

Proof. The properties (1.6) of P imply that the assignment X — > X P defines a map 
Z — > R. Namely, the first of the properties (1.6) implies that the double coset X e Z 
gets mapped to a single coset in A / J . The second property implies that the latter coset 
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belongs to R. Although P is an infinite sum of elements of U(g), by (1.8) the condition 
(c) implies that only finitely many summands of are non-zero cosets in A / J . 

Let us denote by n* this map from Z to R. Further, denote by 7r* the restriction to 
R of the projection map n : A/ J — > Z. The first property in (1.7) implies the equality 

XP = X for any X G R. (1.11) 

Hence the composition n*n* is the identity map on R. The equality (1.11) also implies 
that 7r* is a homomorphism of algebras. Finally, the second property in (1.7) implies 
that the composition tv^tv* is the identity map on Z. Thus we get (ii), and hence (i). □ 

Thus the algebra Z contains as a subalgebra a copy 7r(R) of the algebra R. It also 
contains as a subalgebra a copy of another Mickelsson algebra, defined as a quotient of 
normalizer of the left ideal J' C A relative to this ideal. However, the latter quotient 
algebra will not be used in this article. Note that both algebras R and Z act on the 
quotient vector space A / J . Here R acts via left multiplication from the algebra A, while 
the action of Z is defined by (1.10). The proof of Proposition 1.1 also demonstrates that 
the isomorphism rr : R — > Z identifies these two actions. 

The algebra Z contains U(t) as a subalgebra and is free both as a left and as a right 
module over it. Further, there is a weight decomposition of Z relative to the adjoint 
action of t on Z, similar to (1.9). The zero weight component Z° is a subalgebra of Z. 
The algebra Z° has several natural subalgebras. One of them is the image 7r(R°) of the 
zero weight component of the algebra R . Further, consider the image of the zero weight 
component A of A C A relative to the canonical projection A — > Z. This image is a 
vector subspace in Z . Denote by S the subalgebra in Z generated by the elements of this 
subspace. Then 

7t(R°)cScZ . (1.12) 

1.3. Algebraic group action 

Let G be a reductive algebraic group over C with the Lie algebra g. Suppose there is 
an action of the group G by automorphisms on the algebra A, extending the adjoint 
action of G on the subalgebra U(rj) c A. Suppose that the action of G on A preseves 
the subspace V, and that the action of G on V is locally finite. Moreover, suppose that 
the action of g on A corresponding to that of G coincides with the adjoint action: 

X : adx(A) = XA- AX for iGg and Aek. 

Let Go be the connected component of G and T C Go be the maximal torus of G 
with the Lie algebra t. Let NormT and NormoT be the normalizers of T in G and Go 
respectively. The adjoint action of the group Norm T on t identifies the quotient group 
Norm T/T with the Weyl group & of g. Choose a representative o c G Norm T of o c . 
The elements d\ , . . . , ay G G can be chosen to satisfy the braid relations 

g c d d d c . . . = d d etc &d ■ ■ ■ for c^d (1-13) 

m cd m cd 

where m c d is the order of the element o c o~d G & , see [T] . For any reduced decomposition 
o~ = cr Cl . . . a Ck in & put 
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a = a Cl ... a Ck . 

Due to (1.13) this definition of a is independent of the choice of the decomposition of a . 
Thus we get an action of the braid group of $j by automorphisms a of the algebra A. 

The adjoint action of the group Norm T on t identifies Norm T / T with a semidirect 
product SH = Six6 where X is a certain finite subgroup of Norm T / T . Each element 
t G X acts on t by permuting Hi, . . . ,H r . Hence r determines an automorphism of 
the semisimple part of g , which permutes Ei , . . . , E r and in the same way permutes 
Fi , . . . , F r . For each r G X choose a representative r G Norm T whose adjoint action on 
the semisimple part of $j yields this automorphism. Then f 4t and a i— > a under the 
canonical map 

NormT -> NormT/T = 9t. 

Moreover, the representatives a± , . . . , a r above can be chosen so that the adjoint action 
of t on G permutes them in the same way as r permutes Hi , . . . , H r . 

Regard q as a subspace of A . Note that because the action of a G G on the subspace 
t C A factors through that of a G 6 , the action of a on A maps any vector of weight 
A to a vector of weight cr(A) . Let A C A be the zero weight subspace relative to the 
adjoint action of t. This subspace is preserved by the action of all the elements <7,r G G. 
Since the subgroup TcG acts on A trivially, we get an action of the group D\ on A . 

Since the action of the elements cx,t G G on A preserves the set of denominators (1.3), 
it extends from A to A. Further, the action of f on A preserves the ideals J and J'. 
Hence f acts on A / J and on Z . Similarly, r acts on A and on Z . 

1.4. Zhelobenko operators 

For each c = 1 , . . . , r define a linear map £ c : A — > A by setting for any A G A 

00 

£ C (A) = A + {slH c (H c -l)...(H c -s + l)y 1 E^d s Fc (A) (1.14) 

s = l 

By (c) for any given A G A only finitely many terms of the sum (1.14) differ from zero, 
hence the map £ c is well defined. The definition (1.14) and the next two propositions go 
back to [Zl, Section 2]. See [KN1, Section 3] for detailed proofs of these propositions. 

Proposition 1.2. For any H G t and A G A we have 

UHA) G (H + a c (H)) £ C (A) + J , 

£ c (AH)e £ c (A)(H + a c (H)) + J. 

Proposition 1.2 allows us to define a linear map ^ c : A — > A/ J by setting 

t c (XA) = Z£ C (A) + J for left) and A G A, 

where the element Z G U(t) is defined by the equality Z(pt) = X(p + a c ) for [i G t* 
when X and Z are regarded as rational functions on t* . 

The action of the Weyl group & on the algebra U(t) extends to an action on U(t) , 
so that for any a 66 

(aX)( l ,) = X(a- 1 ^)) 

if X G U(t) is regarded as a rational function on t* . Take the images a c ( J) and a c ( J') 
relative to the action of d c G G on A ; they are respectively right and left ideals of A . 
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Proposition 1.3. We have <x c (J) C Ker £ c and £ c (&c(J')) C J'+ J. 

This proposition allows us to define the linear maps 

£ c :A/J->A/j and £ c : Z ->■ Z 

as the composition £ c cr c applied to elements of A which are taken modulo J and J + J' 
respectively. In their present form, the maps £ c have been introduced in [KO]. The next 
proposition states their key properties. For the proof of Part (i) see [Zl, Section 6]; for 
the proofs of (ii) and (iii) see [KO, Section 5]. 

Proposition 1.4. (i) The maps £i,...,£ r of A/ J and Z satisfy the braid relations 

£ c £d £ c 
N 

m cd 

(ii) The map £ c of Z is an automorphism of the double coset algebra: 

UX*Y) = UX)oUY) for X,YeZ. 

(iii) The map £ c of A / J is Z -equivariant in the following sense: 

£ C (X oZ)= £ C (X) o tc(Z) for X G Z and ZgA/J. 

Part (i) of Proposition 1.4 implies that for any reduced decomposition of an element 
a = <t Ci . . . a Ck in the Weyl group & the map 

(* = (<*■■■ (c h (1-15) 

of A / J or Z does not depend on the choice of the decomposition. In view of Part (ii) , 
any map £ CT of Z will be called a Zhelobenko automorphism of Z. Any map £ CT of A/ J 
will be called a Zhelobenko operator on A/ J. Then using the same symbol £ CT for the 
maps of both A/ J and Z should cause no confusion. By Proposition 1.2 

Z a (HZ) = (<ToH)Z <T (Z), (1.16) 
UM)=UZ)(aotf) (1.17) 

for any a G 6, -H" G t and ZeA/JorZGZ. Here for c = 1 , . . . , r we use the equality 

a c (n + a c ) = a c o n for ^Gt*. 

The relations (1.16) imply that all the operators £ CT on Z preserve the subspace Z . 

We will complete this subsection with an explicit formula for the operator £ c on Z. 
Let us denote by q c the sl2-subalgebra of q spanned by the elements E c , F c and H c . 
It acts on the vector space A, by restricting to q c the adoint action of g on A. As 
an rj c -module, A splits into a direct sum of irreducible finite-dimensional submodules. 
Choose j E {0, 2i 1, • • •} and take any element A e A from an irreducible rj c -submodule 
of dimension 2j + 1 . Suppose that A is of weight 2 /j relative to the action of H c , that 
is [i?c,A] = 2hA for h G + l,...,j — Since a c is an automorphism of 

the algebra A, the element a c (A) G A belongs to an irreducible c -submodule of the 




for c 7^ d. 



tn cd 
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same dimension 2j + 1 , and is of weight — 2h relative to the action of H c . Therefore the 
operator £ c on Z maps the coset of A to the coset of 

oo 

{slH c (H c -l)...(H c -s + l)y 1 E: a d s Fc (Z c (A)) = 

s=0 
j-h 

{s\H c (H c -l)...(H c -s + l)) _1 i? c s ad^ c (a c (A)) . 

By the definition of subspace J C A, the coset of this sum coincides with the coset of 
j-h 

£ i slH c(H c -l)...(H c -s + l)) _1 ad^ c ad s Fc (d c (A)) ■ 

s=0 

But the coset of the last sum in Z coincides with the coset of 

\ 11 (i + l)(H c -i) aMh (L18) 

s=0 i=0 \ < J \ c ) 

see for instance the proof of [KN1, Proposition 3.7]. The sum over s = 0, . . . , j — h of 
the products of fractions in (1.18) is a particular value F(h — j ,j + h + 1, —H c ; 1) of 
the hypergeometric function. Using the well known formula 

T(w)T{w -u-v) 

F(u,v,w; 1) = 



T(w — u) Y(w — v) 

valid for u,v,w G C with w^O, —1, . . . and Re (w — u — v ) > 0, the sum (1.18) equals 

T(-H c )T(-H c -2h-l) ^ _ H c + 2h + i + l ^ 

Y{-H c -h + 3 )Y{-H c -h-j-l) ac{A} ~ l\ H c - l + l ac{A} 

j-h j-h 

= WiHc-i + l)- 1 -d c {A)- l[(H c + i + l). 
1.5. Invariants of Zhelobenko automorphisms 

Consider the subspace A G of G -invariants in A. Define the linear map 7 : A G — > Z 
as the restriction to A G of the canonical projection A — > Z . The natural embedding of 
algebras A — > A determines an embedding of U(t)-bimodules Z — > Z. Consider the zero 
weight component Z° of the vector space Z. Here we again refer to the adjoint action 
of t on Z. The definition of the action of £ CT and f on Z implies immediately that the 
image of the map 7 is contained in the subspace of Z , 

Q = { Z G Z° I £ a (Z) = Z and ?(Z) = Z for all a G 6 and r G 1 } . 

By [KNV, Remark 3] this Q is a subalgebra of Z, and is included in the chain (1.12): 

Q C tt(R°) c S C Z°. (1.19) 

The next proposition has been also proved in [KNV] . 
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Proposition 1.5. The map 7 : A G — > Z is injective, and its image is equal to Q. 

The squares of the Zhelobenko automorphisms |i , . . . , | r of Z are given by the formula 

H(Z) = (H c + 1) ol(Z) (H c + l)" 1 for all Z G Z, 

see [KO, Corollary 7.5]. Here cr c 2 G T by definition, so that the squares cr c 2 and hence £ c 2 
act trivially on the zero weight subspace Z° C Z. This means that the restrictions of 
all the operators £ CT to Z° determine an action of the Weyl group &. Furthermore, for 
any r G X the adjoint action of r G G on g permutes the operators £1 , . . . , £ r on Z in 
the same way, as it permutes the elements Hi , . . . , H r of t. This implies that the latter 
action of the group 6 on Z° extens to that of the semidirect product SH = I k 6. 

We call a weight p, G t* nonsingular if p(H a ) 7^ —1, —2, . . . for all a G Fix any 
A G t* such that A + p is nonsingular. Let N be any irreducible S-module of weight A. 
The latter condition means that the subalgebra U(t) C S acts on N via the mapping 
A : t — > C. This mapping extends to a homomorphism U(t) — > C, also denoted by A. 

Due to (1.19), N is a module over the algebra Q by restriction. Let &\ and £Ha denote 
the stabilizers of A G t* in D\ and 6 relative to the shifted actions of these groups on t* ; 

6 A = { a g 6 I (to A = A}. 

The shifted action of & on t* extends to an action of 5^ since r(p) = p for every r G X. 

Proposition 1.6. Suppose that A + p G t* is nonsingular, and that D\\ = &\ . Then N 
is an irreducible Q-module. 

We shall prove Proposition 1.6 in the remainder of this subsection. Let x ■ S — > End N 
the defining homomorphism of the S-module N. For the proof of the proposition it is 
sufficient to find for each Y G S an element Z G Q such that x(X) = x(Z)- Here we 
will assume that Y is the image of some element A G A under the canonical projection 
A — > Z ; see the definition of the subalgebra S given at the end of Subsection 1.2. 

Lemma 1.7. There exists X G U(t) snc/i t/iat A(X) 7^ and for any a G © t/ie element 
^ a {XY) G Z belongs to the image of A under the canonical projection A — )■ Z. 

Proof. All elements a (A) with cr G © belong to some finite-dimensional submodule M 
of A under the adjoint action of g. For every positive root a G A + we can choose a 
non-negative integer n a such that 

ad£; +1 (B) = for all BeM. (1.20) 

Put 

x= n n^« + ^«) +s )- ( l21 ) 

a£Zi+ s=l 

Then we have A(X) 7^ 0, because the weight A + p is nonsingular by our assumption. 
Now take any element a G & and a reduced decomposition a = a Cl . . . a Ck . Denote 



ui = a Cl . . .a Cl and fii = ui-\{a Cl ) for l = l,...,k. 
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Note that $\ , . . . , are all those roots a G A + for which a -1 (a) ^ A + . By (1.16), 

i a (XY) = (aoX)i a (Y). 

In each factor £ C; of ^ a in the product (1.15) we can replace, modulo the left ideal J' of 
A, every entry of E C[ by the corresponding adjoint operator. Using the property (1.16) 
repeatedly, in £, a (Y) we can also move to the left all denominators from U(t). Hence 
£cr(F) equals to the coset in Z of the image of a (A) under the operator 

oo k 

Yl II ( s < ! oH c l )--- (wi-i o H Cl - Sl + l))" 1 • ad^ad^ . . . ad% fc ad% fc . 

si,...,s fe =0 1=1 

By the assumption (1.20), all summands above with at least one index si > np l vanish. 
Further, for all / = 1 , . . . , k and s = 1 , 2 , . . . we have 

u; z _! oH Cl -s + l = H Pl + (u^_\(p) - p){H Cl ) - s + 1 

= H f , l +p(H fil )-l-s + l 

= H Pl +p{H Pl )-s. (1.22) 

Now for I = 1 , . . . , k consider the factor in (1.21) corresponding to the positive root 
a = —a~ 1 (/3i) and an index s = 1 , . . . ,n a . The shifted action of a on this factor yields 

a o H a + p(H a ) + s = -H Pl + (a _1 (p) - p){H a ) + p(H a ) + s = -H Pl - p{H Pl ) + s. 

The yielded factor cancels, up to the minus sign, the factor (1.22) in the denominator 
of £, a (Y) . Hence the product (a o X) £a(Y) G Z belongs to the image of A . □ 

Take the element X e U(t) from Lemma 1.7. We have = X(X) x(X) because 

the S -module is of weight A by our assumption. Hence 

X(Y) = X (XY/\(X)). 

Replacing the given element Y G S by X Yj X(X) , we may from now assume that for any 
a G & the element ^cr(^) G Z belongs to the image of A under the projection A — > Z. 
Consider the orbit of the weight A relative to the shifted action of the group D\ on t*, 

O x = {u o A | u G 01} . 

This is a finite subset of t*. Hence there exists an element X' G U(t), such that X(X') = 1 
and p(X') = for any weight p G 0\ with p ^ X. Put 

Z = \& X \- 1 J2L(X'Y). (1.23) 

Here ^ denotes the composition r £ a of operators on Z, if ui = r a for some a G & and 
r G X. Due to our assumptions on the given element Y, we have Z G S. Moreover, for 
every u G SH we have = Thus Z G Q. Let us show that xOO = x(%)- 

Until the end of this section, the symbol = will indicate equalities in the algebra Z 
modulo the right ideal generated by all the elements H — X(H) with H G t. Firstly, take 
any u G %K such that u o X ^ A. By our choice of the element X 1 ', then we have 

Ux'y) = ( W ox')| w (y) = \(uox')i u (Y) = (u- 1 oX)(x')^(y) = o. 

Hence x(iw(X'Y)) = unless u G d\\ = &\. Let us now prove another lemma. 



Mickelsson algebras and representations of Yangians 



19 



Lemma 1.8. For any a G &\ we have £ a (Y) = Y . 

Proof. By [B, Proposition V.3.2] the subgroup &\ C & is generated by the reflections 
a a corresponding to the positive roots a G A + such that (X+p)(H a ) = 0. Fix such an a 
and write a = a(a c ) where a c is a simple root and a G &. Since Y G Z , then we have 

U(y) = (|«r|c|a-i)(y). 

Consider the element Y - ' = £ CT -i(Y) of Z. By our assumptions on Y, the element 
y' is the image of some element A' G A under the projection A — > Z. Consider the 
s [2 -subalgebra g c Cg. First suppose that A' belongs to an irreducible g c -submodule of 
A. If A' 7^ 0, then this submodule has an odd dimension, say 2j + 1 . By the calculation 
made in the end of Subsection 1.4, the operator £ c maps the coset of A' in Z to 

J II, - / • 1 



1 J U. H c -i + l 



Here we also used the following observation: because A' G A belongs to an irreducible 
fj c -submodule of dimension 2j + 1, we have a c (A') = (— 1) J A'. Thus 



= (-d j n r ■ ^p") = (-d j n - : H c : ■ ^ 

- LJ - a o ii„ — z + 1 - LJ - a o _H r — z + 1 

i=l i=l c 

But 

A(a o ff c ) = (a" 1 o A)(^ c ) = (a" 1 (A + p) - p)(H c ) 

= (A + p)(a(tf c )) - p(a(if c )) = (A + p)(H a ) - 1 = -1 , 

so that 

i=l 

The assumption that A' belongs to an irreducible g c -submodule can now be removed 
without any loss of generality. Lemma 1.8 is thus proved for any a = o a . □ 

Using Lemma 1.8, for any a G &\ we now get 

ia(X'Y) = (<roX')£ a (Y) =X(aoX')Y= (a -1 o X)(X') Y = X(X') Y = X'Y . 
This completes the proof of the equality x (Y) = x (Z) and hence that of Proposition 1.6. 
1.6. Irreducible S -modules 



In this subsection we will introduce a class of irreducible S-modules, to which we can 
then apply Proposition 1.6. Let K be any left A-module. Since U(g) is a subalgebra of 
A, we can regard K as a g-module by restriction. Suppose the action of the subalgebra 
t C Q on K is semisimple, so that K splits into direct sum of the weight subspaces K x 
where A ranges over t*. Also suppose the action of the subalgebra n' C g on if is locally 
nilpotent, so that for any v G K there is a positive integer s such that (n') s v = {0}. 
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Consider the space K n of n-coinvariants of K. Take any A G t* and consider the weight 
subspace K x C K n . Denote by N the subspace of K consisting of all n'-invariants of 
weight A . By restricting the canonical projection K — > K n to the subspace N C K , we 
get a natural linear map N — > K x . In general, the restriction map may be not bijective. 

Now suppose that the weight A + p is nonsingular. It turns out that then the map 
N — > K x is bijective. To define the inverse map, take any coset / G K x and choose its 
representative u G K. We may assume that u G K x . Then we can define a linear map 
K x — > N by mapping / i-> Pu. Here we use the properties (1.6), the second property in 
(1.8), the local nilpotency of K relative to n', and the nonsingularity of X + p. This map 
is the right and left inverse to the projection N — » K x , due to the first and the second 
properties in (1.7) respectively. See also the proofs of the properties (1.6) we referred to. 

Since A + p is nonsingular, the weight subspace K x C K n has a structure of a module 
over the algebra S. The action of S on K x will be denoted by o like in Subsection 1.2, 
and defined by 

Xof = XPf for XeS and / G K x . 
At the right hand side of the above equality, we use the action of the algebra Aonif. 

Proposition 1.9. Suppose that K is an irreducible A-module, while the weight X + p is 
nonsingular. Then K x is an irreducible S -module. 

Proof. Take any two cosets /, # G K x and choose their representatives u , v G K . We 
may assume that both u,v G K x . Determine the element Pu G K x as above. Because 
K is an irreducible A-module, we can find an element A G A such that APu = v. Let 
X G Z be the coset of A . Then we have IeS and X o / = g . □ 

2. Shapovalov forms 

2.1. Howe systems 

Suppose that the reductive Lie algebra q is symmetric. That is, q is equipped with an 
involutive automorphism identical on the Cartan subalgebra t and preserving each of 
the nilpotent subalgebras n and n'. We have a Cartan decomposition g = q + ©g_ where 
q+ and Q- are the eigenspaces of this automorphism with the eigenvalues 1 and — 1. 
Then g + is a Lie subalgebra of g containing t, while 

[s+,6-]CS- and [g_,0_]C0+. 

Note that then the involutive anti-automorphism e of q preserves the decomposition 
g = + © Q- . This decomposition induces decompositions of the nilpotent subalgebras: 
we have n = n + © n_ and n' = © where 

n + = n n g + , n_ = n fi g_ , n' + = n' (1 g + , n'_ = n' n g_ . 

Let U be any finite-dimensional complex vector space and 9 G {1,-1}. Depending 
on whether 9 = 1 or 9 = —1, denote by H({7) the Weyl algebra or the Clifford algebra 
of U © U*. This is the complex associative unital algebra generated by the elements of 
U and of the dual space U* subject to the relations for «,o6U and u' ,v' G U* 
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vu = 9uv, v'u' = 9u'v', u'u — 9 uu' = u' (u) . (2-1) 

Equip the vector space W = U © U* with the bilinear form B , symmetric if 9 = — 1 
and alternating if 9 = 1, such that B(u,v) = B(u',v') = and B{u,u') = u' (u) . The 
relations (2.1) are then equivalent to the relations for all w,w' E W 

w'w -9ww' = B(w,w'). (2.2) 

Then any isotropic subspace V C W generates a subalgebra of H (£/"), which is a free 
commutative algebra if 9 = 1, or free skew-commutative if 9 = — 1. This subalgebra 
will be denoted by P(V). In particular, the algebra H(£/") contains two distinguished 
subalgebras P (U) and P (£/"*) , generated by the elements of U and of U* respectively. 

Further, equip the vector space W with a grading so that the direct summands U 
and U* of W have degrees 1 and —1 respectively. This grading naturally extends to 
a Z-grading on the algebra K(U). In particular, the subalgebras P (U) and P (U*) of 
H(£7) are graded respectively by non-negative and non-positive integers. 

Now suppose there is a homomorphism of associative algebras C : U(g) — > B.(U). 
Further suppose there is an action of the group G on the vector space W preserving the 
bilinear form B. Since the form is preserved, this action extends uniquely to a action 
of G by automorphisms of the algebra H(L r ) . In particular, we get an action of the Lie 
algebra q on H(L r ) , and the weight decomposition 

H(U) = © H(U) X . (2.3) 
\et* 

Choose a non-degenerate symmetric bilinear form ( , ) on the vector space U . Using 
this form, define a linear map e : W — >■ W such that e : U — >■ U* , e : U* — >■ U while 

e(u)(v) = (u,v) and (u,e(u')) = u' (u) (2.4) 

for any u,v G U and u' EU*. The map e is involutive and for all w ,w' E W satisfies 

B(e(w),e(w')) = -9B(w,w'). 

Therefore this map admits a unique extension to an involutive anti-automorphism of 
the algebra H(U). We denote the extension by e again. 

We shall say that that the homomorphism ( : U(g) — > H(£7) and the action of G on 
W form a Howe system on U if the following six conditions are satisfied: 

(1) the map ( : U(g) — > H(U) is G-equivariant; 

(2) the action of g on H(£7) corresponding to that of G is adjoint to £; 

(3) e(((X)) = ((e(X)) for all X E £j, where e : g — > g is the Chevalley anti-involution; 

(4) C(n) C U-R{U) and C(t) C C- 1 + U-R{U); 

(5) [C(g+),U] C U and [((n_) ,U*} C U while [C(n_),tf] = {0}; 

(6) any weight element of P(£7) has a Z-degree uniquely determined by the weight. 

The property (2) of a Howe system means that the action of Lie algebra g on H (C7) 
corresponding to that of G is given by 



X :Y^[C(X),Y] for XEg and YeR{U). 
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The properties (2) and (3) imply that the automorphism e of H (U) changes the signs of 
the degree and of the weight. Here and in (6) we refer to the Z-grading on H(C7) and the 
weight decomposition (2.3). By combining (3) and (4), (5) we get two more properties: 

(7) C(ti') CH(£/)- U* and C(t) C C • 1 + H (U) ■ U*; 

(8) [{(g+),U*] C U* and l((n'_),U] C U* while [((n'_) ,U*} = {0}; 

The properties (5) and (8) imply that for X G g + ,n_,n!_ the adjoint operators ad<^x) 
on H(C/) have the degrees 0,2,-2 respectively. 

Later on we will work with known examples [HI] of Howe systems. The list of (1) to (8) 
summarizes the common properties of these examples that we shall use. Now fix any 
Howe system on U and set 

A = U(g)(g)H(L/). (2.5) 

The group G acts (diagonally) by automorphisms of the algebra A. Let us identify U(g) 
with subalgebra of A generated by the elements 

I®1 + 1®((!) where X E q. (2.6) 

Then set 

V = l<g>H(tf). 

The condition (2) on a Howe system imply that the condition (c) on the algebra A is 
satisfied. Note that in this case V is a subalgebra of A, not only a vector subspace. In 
this section we will investigate the Mickelsson algebras R and Z corresponding to the 
associative algebra (2.5) determined by any Howe system. We will usually identify any 
element X G g with the element (2.6) of A, and any element Y G H({7) with 1 eg) Y G A. 
Note that for X G g the commutator [X, Y] in the algebra A equals Y] G H(£Z) . 

2.2. Shapovalov form on double coset algebra 

Let us extend the map e from H({7) to the algebra A defined by (2.5), so that under 
the extension X <g> Y (->■ e(X) <g> e(Y) for X G U(g) and Y G H(£/)- Here the symbol 
e in the first tensor factor denotes the Chevalley anti-involution on U(g) . The same 
symbol e in the second tensor factor denotes the involutive anti-automorphism of H(i7) 
defined by (2.4). The extended map is an involutive anti-automorphism of the algebra 
A. Its restriction to the subalgebra U(g) generated by the elements (2.6) coincides with 
the Chevalley anti-involution on U(g) , due to the property (3) of a Howe system. This 
anti-automorphism further extends from A to A. 

Since the latter extension preserves the subspace J + J ' C A , it defines an involutive 
linear map e : Z — > Z. Because s(P) = P where P is the extremal projector for g, this 
map is an anti-automorphism of the double coset algebra Z. We will denote it by the 
same symbol e. Clearly, e maps the subalgebra S C Z to itself. Denote by K the right 
ideal of the algebra Z, generated by the cosets of all elements u G U. Similarly, denote 
by K' the left ideal of the algebra Z, generated by the cosets of all elements u' G U*. 
We identify the elements u,u' G H(L r ) with the elements 10«,18m'6A respectively. 

Lemma 2.1. (i) K is spanned by the cosets of products uX in A where u G U , X G A . 
(ii) K' is spanned by the cosets of products Xu' in A where u' G U* , X G A. 
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Proof. Take any u G U and I 6 A. The property (5) of a Howe system implies that 
for any A G n the commutator [it, A] G £7. We may assume that u is a weight element, 
say u e U x for some A G t* . If A G n is also a weight element then [u , A] has a weight 
less than A . The inequality \i < A for some fj, G t* means that the difference A — is a 
non-zero sum of simple roots in A + with non-negative integral coefficients. By the first 
relation in (1.7), 

uPX G uX + J2 vY + J ( 2 - 7 ) 

where v ranges over a certain finite subset of U and has a weight /i < A , while F 6 A 
corresponds to v. But (2.7) shows that the product of the cosets of u and of X in the 
algebra Z is the coset of 

uX + J2 vY - 

v 

Furthermore, we can recursively invert (2.7) and get the relation in the algebra A, 

uX G uPX + wPZ + J 

w 

where w ranges over some finite subset of U and has a weight less than A , while Z G A 
corresponds to w . So the coset in Z of the product «X in the algebra A is the coset of 

uPX + wPZ 

w 

and therefore belongs to the ideal K of the algebra Z . This completes the proof of the 
part (i) of the lemma. The proof of the part (ii) is very similar, and is omitted here. □ 

Corollary 2.2. For any Z G Z there is a unique element (Z) G U(t) such that 

Z-(Z) G K + K'. 

Proof. Choose any ordered basis in each of the vector spaces n,n' ,U ,U' and t. By the 
classical Poincare-Birkhoff-Witt theorem, each of the vector spaces U(n) ,U(n') ,P (U), 
P (U*) and U(t) has a basis made of all monomials in the corresponding basis vectors. If 
A,A',B,B' and C run through these monomials, then the products ABCB'A' form a 
basis in A. So the vector space Z is spanned by the cosets of products in A of the form 
BDB' where B and B' are as above whereas D G U(t) . By Lemma 2.1, any such a 
coset belongs to K + K' unless B = B' = 1. Thus (Z)o G U(t) exists for any Z G Z. 

Now suppose that (Z) G U(t) is not unique. Then Lemma 2.1 implies the existence 
of a nonzero element of U(t) which belongs to the sum of the right ideal of A generated 
by n and U, and of the left ideal of A generated by n' and U*. But this is not possible 
due to existence of the basis in A described above. □ 

Now regard Z as a left U(t)-module. Define the Shapovalov form S : Z x Z — > U(t) by 
S(X,Y) = (e(X)oY) for X,YeZ. 



This form is symmetric, contravariant and U(t)-linear by definition. That is, we have 
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Note that 



2. 3. Shapovalov form on coinvariants space 



(2.8) 
(2.9) 
(2.10) 

(2.11) 



S(X,Y) = S(Y,X); 
S(ZoX,Y) = S(X,e(Z)oY) for ZeZ; 
S(ZX,Y) = ZS(X,Y) for ZeU(t). 

S(X, Y)=0 when FgK'. 



Let fi G t* be any weight. Denote by the left ideal of A, generated by the elements 
X of n', and by the elements H — li(H) where H e t. Here we identify any element X 
of g with the element (2.6) of the algebra A. Further, denote by J" the left ideal of A, 
generated by the elements X <g) 1 where l£n', and the elements <g> 1 — jtt(-ff) where 
H £ t. Let I be the left ideal of A generated by the elements of U* C H(CT') . The first 
inclusion in the property (7) of a Howe system implies that X — X <g> 1 e I for Xeti'. 
The second inclusion in the property (7) implies that there is a weight k E t* such that 

((H)- k(H) e~R(U)-U* for all H e t. (2.12) 

Hence H — H ®1 — k(H) e I for H E t. Therefore 

j;+i=Ji+K+i. (2.i3) 

Let A^ be the quotient space of A relative to the left ideal J^' + 1. Let M M be the 
vector space of double cosets of A relative to the left ideal J^' + 1 and the right ideal J : 

A M = A/ (j; + I) , = A M /J = A/(J^'+I + J). (2.14) 

Regard the ring P(U) as a left H(t/)-module, by identifying this ring with the quotient 
of H(U) over the left ideal generated by U*. Thus for X e B.(U) and u,v e P(U) 

X(u) = v if Xu-v eR(U) - U* . (2.15) 

Then we can identify the A-module A M with the tensor product <g> P (U) where M M 
is the Verma module of the algebra U($j) appearing as the first tensor factor in (2.5). 
The vector space M M gets identified with the space of coinvariants 

(M M (g)P(t/)) n . (2.16) 

The Cartan subalgebra t C Q acts on this space via left multiplication in A. Moreover, 
this space is a left module over the Mickelsson algebra R. It is generated by the image of 

1 M <g) 1 e M M <8) P (U) (2.17) 

where 1 M is the highest weight vector of the Verma module M M . By using the equality 
(2.13), any element H e t acts on the vector (2.17) as multiplication by (fx + k)(H) . 
Hence the image of the vector (2.17) in belongs to the weight subspace M^ +K . Due to 
the condition (6) on a Howe system, the subspace M^ +K C M M is one-dimensional, and 
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is spanned by the image of (2.17). Note that there is an isomorphism of vector spaces 
P (U) — > M M . It can be defined by assigning to any u G P (U) the coset of 1 <8> u G A in 
the quotient = A/ (J^ + I + J) . This isomorphism will be denoted by . 
Now consider the left ideals of the algebra A , 

j; = u© j; ^d i=u(t)i. 

Suppose the weight |i + KGt* is generic, that is (ix + K,)(H a ) Z for all a G A + . Then 
the spaces of double cosets A/ (J" + I + J) and A/ (J'' + I + J) = are naturally 
isomorphic. Here we once again use the equality (2.13). Further, M M is isomorphic to 
the left Z-module, defined as the quotient of the algebra Z by the left ideal generated 
by the cosets of the elements of 1 <g) U* and the cosets of the elements H — (fi + k)(H) 
where H G t. Here we use the isomorphism of the algebras R and Z; see Proposition 1.1 
and remarks made immediately after stating it. Accordingly, we will use the symbol o 
to denote the action of the algebra Z on . Then any element of M M can be presented 
as Z o t M (l) for some Z G Z. Since t M (l) G M» +K , we get Z o t M (l) G if Z G Z^ and 

i/ = A-//-«. (2.18) 

We may also choose Z from the image in Z of the subalgebra 1 <g) P(C/) C A. Indeed, 
due to (1.7) the cosets of (1 <g> tt) P and 1 ® w in M M coincide for any element u G P (C/) . 

Due to (2.8)-(2.11), for a generic weight /i + k the Shapovalov form 5 1 : Z ® Z — > U(t) 
defines a symmetric contravariant form 5^ : ® M M — >■ C by setting 

S fl (XoL fl (l),Yo^(l)) = S(X,Y)(fi + K) for I,YeZ. (2.19) 

Here S(X, Y)(/a + k) is the evaluation at fx + k of an element of U(t), regarded as a 
rational function on t*. The contravariance of the form means the equality 

S»(Zof,g) = S^f,e(Z)og) for f,geM^ and ZeZ. (2.20) 

Note that if / = l^u) and g = i^v) for some u and v from P (U) , then 

S lt (f,g) = S(X,Y)(vL + K ) (2.21) 

where X and Y are respectively the images of 1 ® it and 1 <S> v under the projection map 
A — > Z . This follows from the remark made at the end of the previous paragraph. 

In Subsection 2.1 we selected a non-degenerate symmetric bilinear form ( , ) on the 
vector space U . Let us now extend this form from U to P (U) in a natural way, as follows. 
Choose an orthonormal basis u\ , . . . , u n of U. Thus (it,, Uj) = dij for % ,j = 1 , . . . , n. Put 

n 

«...<",<...<") = ]JS Pkqk p k \ 

k=l 

where for every k = 1 , . . . , n we take Pk,Qk G {0,1,2,...} in the case 9 = 1, whereas 
Pk , Qk G {0 , 1} in the case 9 = — 1 . This form on P (U) is uniquely determined by setting 
( 1 , 1 ) = 1 and by declaring that the transpose to the operator of left multiplication by 
any u k is given by the action of u' k G U* . Here u[ , . . . , u' n is the basis in U* dual to the 
basis iti , . . . , u n in U . Thus for each k = 1 , . . . , d we have 
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(ukU,v) = {u,u' k v) for u,v &P(U) . 

Here we regard P(U) as a left H(£/")-module, see (2.15). Since u' k = e(uk) , the above 
displayed equality implies a more general equality, 

(Xu,v) = (u,e(X)v) for u,veP(t/) and XeK(U). (2.22) 

Proposition 2.3. Let \x + k G t* be generic. Then for any u,v G P(U) 

S»(f,g) = (u,C(P\m + p])v } if f = i^{u) and g = i fl (v). 

Proof. By definition, the extremal projector P is a product in U(g) over the set A + of 
positive roots equipped with any normal ordering. Any such an ordering on A + has the 
following basic property [AST] . Take any a , G A + such that [E a , Fp] ^ 0. Then a — (3 
is a root. The commutator [E a ,Fp] is proportional to E a _p or to Fp_ a respectively if 
a — (3 G Z\ + or /3 — a G A + . Now suppose that the root a precedes /3 in the ordering. 
Then the basic property is that either the positive root a — f3 precedes a , or the positive 
root ft — a is preceded by ft. Using repeatedly this property and the definition (1.4), 
we can present the extremal projector P = P[p] as the sum in U($j) of the products 
of the form AA'C where A and A' are normally ordered monomials in the generators 
F a and E a respectively. Here a ranges over the set A + . These monomials form bases 
respectively in the vector spaces U(n') and U(n). Further, any monomial A A' appears 
in the sum P with a unique factor C G U(t) on the right of it. This C is a ratio of a 
complex number to a finite product of factors 

H a + p(H a ) + s (2.23) 

where a is a positive root and s is a positive integer. We will also regard the elements 
of U(t) as rational functions on t* . By (2.19) the value <S^(/, g) of the Shapovalov form 
equals the sum of the values (X)o(fj, + k) where X is the coset in Z of the product in A, 

l®e{u)- AA'C -l®v. (2.24) 

Without loss of generality we may assume that the element v G P(U) has a weight, 
say A G t*, relative to the adjoint action of t: 

[C(H),v] = X(H)v for all H G t. (2.25) 

Then 

C -\®v = \®v ■ D (2.26) 
where D G U(t) is obtained from C by replacing every factor (2.23) respectively by 

H a + (\ + p)(H a ) + s. 

Consider the product 

l®e(u)-AA' -l®v (2.27) 

in the algebra A. Here A = F a ...Fp and A = E a >...Ep> for some positive roots 
a, . . . ,/3 and a' , . . . ,/3'. By the definition of the ideals J and J' of the algebra A, the 
coset in Z of the product (2.27) coincides with that of the product of commutators 
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[F a ,...,[Fp,l®e(u)]...] ■ [E a ,,...,[Ep,,l®v]...] = 
1 ® [{(F a ) , . . . , [C(Fp) , e(u)} . . .} ■ 1 <g> [C(£?«0 , • • • , [CCM ,«]..•]• 

Denote by F the coset of the last product. Due to the properties (4) and (7) of a Howe 
system, then we have (Y)o = (Z) where Z denotes the coset in Z of the product in A, 

1 ® • C(F a ) . ..{(Fp) ■ C(E a ,) . ..C(Ep,) -v) = l® (e(u) C(AA')v) . 

Note that here (Z)o G C C U(t) . Further, the adjoint action of the subalgebra t C fj 
normalizes the ideals J and J' of A. It also normalizes the ideals K,K' of Z. Hence the 
above argument together with (2.26) implies that (X) = (ZD) Q = (Z)qD. Therefore 

(X) (ii + «) = (Z) • D(n + k) = (Z) ■ C(/i + k + A) . (2.28) 

But by the definitions (2.12) and (2.25) of the weights k and A respectively, for every 
element H e t we have the relations in the algebra H (U) , 

v(k + X)(H) e v ■ (C(H) + X(H)) + H(U) ■ U* = ((H) ■ v + K(U) ■ U* . 

They imply the equality in the left H(Z7) -module ~P(U), 

e(u) ((A A' ) v ■ C(p + k + A) = e(u) ((A A'B) v 

where B e U(t) is obtained from C by replacing every factor (2.23) respectively by 

H a + (p + p)(H a ) + s. 

Since ( 1 , U • P (U) ) = , the right hand side of (2.28) can be now written as 

(l,e(u)((AA'B)v) = (u,((AA'B)v) . 

By using the definition of the element B e U(t), we now get Proposition 2.3. □ 

Consider again the symmetric bilinear form ( , ) on the vector space P (U) . Take the 
action of the Lie algebra g on the vector space H (U) corresponding to that of the group 
G . By the property (2) of a Howe system, this action of g is adjoint to the homomorphism 
C : U(g) — > H (£/"). By the properties (5) and (8), the action of the Cartan subalgebra 
t C g+ C g preserves the subspaces U and U* of H (£/"). By the definition (2.4) for any 
ueU and u' G U* 

(u , e(u') ) = v! (u) = B(u, u') , 

while the action of the group G on the vector space W = U © U* preserves the bilinear 
form B by our assumption. Therefore for any H e t and for any u , u' as above we have 

([{(H), u],e(u')) + {u, e([((H),u'})) = 0. 

Here e is an anti-automorphism of the algebra H (U) obeying the property (3) of a Howe 
system. Moreover, we have e(H) = H for the Chevalley anti-involution e on g. Therefore 

([((H),u],e(u')) = (u,[((H),e(u')})). 

Since any vector of U can be written as e(u') for some u' G U* , the last equality implies 
that the action of the Cartan subalgebra t on P(U) is self-conjugate with respect to 
the bilinear form ( , ) . Therefore, because the form ( , ) on P (U) is non-degenerate, its 
restriction to any weight subspace of P(U) is also non-degenerate. 
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2.4- Shapovalov form on a weight subspace 

Now let p G t* be an arbitrary weight. Take the subspace of = A/(J + + 1) 
consisting of the elements of weight A . The Cartan subalgebra tc g acts on M M via the 
left multiplication in A by the elements of U(rj) C A. In this subsection, we will assume 
that A + p is nonsingular. Then for any / G and Z G Z the element Z o / G 
is well defined. Indeed, the denominators of P = P[p] do not vanish on if A + p is 
nonsingular. In partucular, is a module over the subalgebra S C Z . Let the weights 
A and p vary under the constraint that their difference A — p is fixed. Then for any given 
elements X,Y G Z we can regard S(X ,Y)(p + k) as a rational function of p G t*. Now 
recall the definition (2.18) of the weight v . 

Proposition 2.4. Suppose that the weight X + p G t* is nonsingular. Then for any given 
elements X, Y G Z" t/ie function S(X ,Y)(p + k) of p has only finite values. 

Proof. Let A and B be any representatives in A of the cosets X and Y respectively. 
Then the e(X) o Y is the coset of 

e(A)PB (2.29) 

where P = P[p] GU(rj) is the extremal projector for g. We assume that the elements A 
and B of A also have the weight v relative to the adjoint action of t. 

Any poles of (s(X) oY)q as of a rational function on t* may arise only from the 
denominators of the summands of P[p]. By the definition (1.4), up to non-zero scalar 
multipliers, these denominators are products of linear factors of the form H a + p(H a ) + s 
with s = 1,2, . . . and a G A + . Let us move these denominators to the right in (2.29) 
through the factor B , and evaluate the resulting denominators at p + K. Then the linear 
factors become 

{p + k + p + p){H a ) + s = (A + p){H a ) + s 
which do not depend on p , and are nonzero numbers when A + p is nonsingular. □ 

When the weight p + k is generic, the Shapovalov form on can be defined by 
the equation (2.21) where / = t^u) and g = i^(v) for some u and v from P (U) , while 
X and Y are the images of 1 <g> u and 1 <g) v under the projection A — > Z . The same 
equation (2.21) can now be used to define an S-contravariant form on the subspace 
C M M for nonsingular X + p and any p. Indeed, if here /, g G then X, Y G Z u so 
that Proposition 2.4 applies. Denote by the bilinear form on the subspace C 
defined by (2.21), 

S£ : MjxM^C. 

h 1 M h 1 

The contravariance (2.9) of the Shapovalov form on Z implies that is a contravariant 
form relative to the action of the subalgebra S C Z . Indeed, if p + k is generic then by 
(2.20) for any /,j£Mj and Z G S we have 

S*(Zof,g) = S*(f,e(Z)og). (2.30) 

When the weight A — p is fixed, both sides of this equality are rational functions of p . 
Hence these two rational functions are the same. So the equality (2.30) holds for any p,, 
provided A + p is nonsingular, which is assumed in this subsection. 
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In the next lemma, we regard P (U) C H (U) as subalgebras of A and use the adjoint 
action of t on A; see the condition (5) on a Howe system. We also identify the elements 
of H(E/) with their images in the double coset algebra Z. The weights of these elements 
in Z are taken relative to the adjoint action of t. By the condition (2) on a Howe system, 
these weights are the same as relative to the adjoint action of t on H(£7) . Recall (2.18). 

Lemma 2.5. Suppose that for some f G and Y G P(U*)~ U we have the equality 
Yof = t M (l) in M M . Then for any X G P{U) V we also have (XY)of = i^X) in M M . 

Proof. We have / = t^u) for a certain element u G P(U), which has weight v under 
the adjoint action of t on P(U) . In Subsection 2.1 we equipped the algebra H(C7) with 
Z-grading so that the elements of U and U* have degrees 1 and —1 respectively. Since 
the weights of the elements X,e(Y) and u of H({7) are the same, by the condition (6) 
on a Howe system these elements also have the same (non-negative) degree. Thus 

deg X = — deg Y = deg u . 

Consider the element (XY) of of M M . It corresponds to the product XY P u in A. 
Write the extremal projector P as a sum, like we did in the beginning of the proof of 
Proposition 2.3. Then move the generators F a and E a of g respectively to the left of 
X Y and to the right of u in the resulting summands of X YP u . This procedure shows 
that modulo J + J', the product XY Pu equals the sum of products of the form X'Y'u' 
where 

X 1 G P (U) v ~ a , F'eP^)-""' 5 and u' G P (U) v+a+P 

while a, ft are certain sums of positive roots. 

Suppose that the coset of X'Y'u' makes a non-zero contribution to (XY)of G M M . 
The condition (5) on a Howe system implies that the adjoint operator ad^ Fa ) on H(U) 
either has zero degree, or vanishes on the subspace P(U) . Hence 

degX' = degX. 

Thus the element Y'u' G H({7) has the degree zero. By our assumption, this element 
does not belong to the left ideal H({7) U* C H({7) . It also has a weight relative to the 
adjoint action of t on H (U) . But then the weight must be zero. Indeed, by the condition 
(8) on a Howe system, the adjoint action of t on H(CT') preserves the left ideal H(i7) U*. 
Modulo this ideal, the element Y'u' of H (U) equals a non-zero scalar, which has the 
weight zero. Thus the weight of Y'u' is zero as well. Therefore a = and X' = X . 

The element Y of of corresponds to the product Y Pu in A. Modulo J + J', the 
latter product equals the sum of the products Y'u' G H({7) where Y' and u' are the 
same as above in the particular case when a = : 

Y' G P (U*)-"- 13 and u' G P (U) u+/3 

where /3 ranges over sums of positive roots. Modulo the left ideal H(CT') U* of H (£/"), the 
sum of the products Y'u' equals a scalar. This scalar must be 1 , because Fo/ = t /J (l) 
in M M . Hence the sum of the products XY' u' equals X modulo H(U) U* C H ([/"). □ 
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The contravariance of the form implies that the kernel Ker is an S-submodule 
of . Indeed, let / G Ker and Z eS. Then Zofe Ker , because for any jGMj 

S^Zof,g) = S^f,e(Z)og) = 0. 

Proposition 2.6. Let A + p G t* be nonsingular. Then the quotient space /KerS 1 ^ 
is an irreducible $ -module. 

Proof. Take any / G such that / ^ Ker S 1 ^ , so that the image of / in the quotient 
space /KerS^ is not zero. For any Y G P(C/*) ~ v consider the vector Y o / G M^. 
This vector is well defined, because the denominators of P = P[p] do not vanish on /. 
The weight of this vector is A — v = p + k, while the subspace M^ +K C is spanned 
by the vector t M (l) . Hence the vector Y - o / is a multiple of t^(l) ■ 

Suppose that Y o / = for every Y G P (U*)~ v . The map e provides a bijection 
P(£7*) _l/ — > P(U) U of subspaces of H(U) . Hence for any g G M^ there is an element 
Y G P (U*) ~ v such that g = t At (e(Y)) . If the weight /i + k is generic, then 

Si (f, 9) = S, (/, e(Y) o v (l)) = S M (Y o /, v (l)) = 0. 

By our definition of the form then we must have (f, g) = for all weights /U, not 
only those where p + K is generic. Hence / G Ker , a contradiction. It shows that there 
is Y G P (U*) ~ v such that Y o / ^ 0. Moreover, we can choose Y with Y o / = ^(1) . 

Now take any h G M^ and choose X G P(U) U such that l^{X) = h. By Lemma 2.5, 
then we have 

(XY)of = h. 

Here X Y stands for the coset in Z of the product of X G P (U)" and Y G P (£/"*) in 
the algebra A. This coset belongs to S by definition, because the product of X and Y 
in A has weight zero. □ 

Recall that Q is a subalgebra of S . By using Proposition 1.6, we now get the following 

Corollary 2.7. Suppose that A + p G t* is nonsingular, and that = &x- Then the 
quotient space M^/KerS 1 ^ is an irreducible Q-module. 

2.5. Irreducible Q-modules 

In this subsection we give another construction of irreducible Q-modules, which employs 
Proposition 1.9. For any weight p G t* denote by the quotient of the Verma module 
M M by its maximal proper submodule N^. This quotient is an irreducible g-module. 
Note that P (U) is irreducible as a module over the algebra H (U) . Hence the tensor 
product <g) P (U) is an irreducible module over the algebra A = U(g) <g> H (U) . Recall 
that the action of the algebra H(i7) on the vector space P(f7) is defined by (2.15). 

We can also regard L M <g>P(U) as a $j -module, by restriction from A to its subalgebra 
U($j) . Here g acts on the tensor factor P(U) via the homomorphism £ : U(g) — > H(U). 
Let us show that then the actions on P (U) of the subalgebras t and n' of $j are semisimple 
and locally nilpotent respectively. Indeed, the action of the group G on H (U) is locally 
finite, see Subsection 2.1. Hence the action of q on H(LZ) adjoint to £ is also locally 
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finite, by the property (2) of a Howe system. In particular, the actions of t and n' on 
H(U) adjoint to ( are respectively semisimple and locally nilpotent. So are the actions 
of t and n' on P (U) , by (2.15) and the property (7) of a Howe system. It follows that the 
actions of t and n' on <g> P (U) are also semisimple and locally nilpotent respectively. 

Take any A G t* such that A + p is nonsingular. Consider the space of n-coinvariants 
of the rj-module ® P (U) , and denote by its weight subspace 

(L M ®P(tf))ic(L/i®P (*/))„. (2-31) 
Then is an irreducible S-module by Proposition 1.9. By using Proposition 1.6 we get 

Corollary 2.8. Suppose that A + p G t* is nonsingular, and that D\\ = &\ . Then 
is an irreducible Q-module. 

Let us now regard as the quotient of the vector subspace by its subspace 

Nj = (JV M ®P(l7))i. (2-32) 

We will prove that under certain conditions on A and p , the subspace of coincides 
with the kernel Ker of the Shapovalov form on . Then the quotient of 
coincides with the quotient / Ker . In particular, then these two quotients are the 
same as Q-modules. Observe that for any vector u G P{U) of the weight A — p,, the 
denominators of ((P[p + p])u get evaluated as products of factors of the form 

(A - p)(H a ) + (p + p)(H a ) + s = (A + p)(H a ) + s 

where a G A + and s is a positive integer. These factors do not depend on p . And they 
do not vanish, when the weight A + p is nonsingular. 

Note that here the weight A — p of the vector u G P (U) is taken relative to the action 
of rj on P (U) via the homomorphism £ : U(g) — > H (U) , by using the definition (2.15). If 
we used the adjoint action of q on P (U) as on a subalgebra of H({7) C A, like we did in 
the previous subsection, then the weight of the same vector u would be equal to (2.18). 

Proposition 2.9. Suppose that A + p G t* is nonsingular, and that C{P[p + p]) u 
for some vector u G P(U) of weight A — p . Then 

Nj = Ker flj . (2.33) 

Proof. Consider the vector 1 M <g> u G M M cg> P(£7) . This vector is of weight A. We will 
show that the image t^(u) of this vector in the quotient space 

M^ = (M^P(U)) x n 

does not belong to the sum + Ker S x . This implies the equality (2.33). Indeed, then 

Mj/(Nj + Ker5j) (2.34) 

is a non-zero quotient of 

Mj/Nj = Lj, (2.35) 
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which is an irreducible S-module by Proposition 1.9. So the quotients (2.34) and (2.35) 
of must coincide, and we obtain the inclusion Ker C . The opposite inclusion 
is obtained from the irreducibility of / Ker as an S-module, see Proposition 2.6. 

Let us now assume that (u) + Ker . We shall bring this to a contradiction. 
By our assumption, the element i^u) G equals the sum of an element of (2.32) and 
of an element l^v) G Ker where v G P(U). Let us regard as a g-module, and 
apply the extremal projector P = P[p] to that equality. Then we get a certain equality 
in the vector space M M <g> P (U) . Here we use the second property in (1.6). 

By applying P to the element i^{u) G we get the vector P(l M <g) u) G M^®Y{U) , 
which belongs to 

1„ <g> ({P[» + p])u + (nM M ) <g> P (U) . 

Here C(-P[a* + 7^ 0. By applying P to any element of (2.32) we get a vector from 
the subspace 

P(iV M <g)P(£/)) C N^®V{U) C (nM M )(g)P(£/). 

By applying P to the element << M (i>) G we get the vector P(l At <8> v) G (g) P (£7) , 
which belongs to 

1„ ® C(^[A» + p])H(nM>P([/). 

But since t M (f) G KerS 1 ^, here we have C(-P[a* + p])f = by Proposition 2.3. We use 
the non- degeneracy of the restriction of the form ( , ) to any weight subspace of P (U) , 
proved in the very end of Subsection 2.3. Thus we get a contradiction. □ 

Later we will produce vectors u satisfying the conditions of Proposition 2.9, for each 
example of a Howe system and for certain weights fx; see Propositions 3.5, 4.4 and 4.6. 

3. Intertwining operators 

3.1. Zhelobenko operators and left ideals 

The vector space W = U © U* has been equipped with a non-degenerate bilinear form 
B, symmetric or alternating. For each isotropic subspace V C W denote by Iy the left 
ideal of the algebra A generated by the elements of 1 <S> V . Also consider the left ideal 

! v = U(t)Iv 

of algebra A. It is generated by the elements of the subspace 1 <g> V C A. In particular, 

lu* = I and If/* = I . 

In Subsection 2.1 we assumed that the group G acts on the vector space W and preserves 
the bilinear form B. Thus a(V) is an isotropic subspace of W for any a G 6. 

Lemma 3.1. For any element a G & the operator £ CT on A/ J maps 

(I + J)/J (T 5(f/ *) + J)/J. 
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Proof. We use the induction on the length £(a) of the element a G & . For £(a) = the 
statement to prove is tautological. Take any index c G {1 , . . . , r} with £(a c a) = 1(a) + 1 . 
Then £o- c o- = £ c £<x. Note that here we have a~ 1 (a c ) G A + . Take any element X G I. 
By the induction assumption, the operator £ CT maps the coset of X in A/ J to that of 

n 

Y k -l®d{u' k ) 

k=i 

where basis elements of U* and Yi , . . . , Y n are certain elements of A . By 

definition, the operator £ c maps the latter coset to 

n 

fc=i 

Due to the definition (1.14), for making the induction step it now suffices to show that 

[F c ,l®a c a(U*)] C l®a c a(U*). 
Applying to this relation the automorphism (d c a) -1 of the algebra A , we get the relation 

[E a ,l®U*]dl®U* for a = a~ 1 (a c ) G 

which holds by the condition (8) on a Howe system. Here (cx c a)~ 1 {F c ) is a multiple of 
E a , because the action of Norm T on the subspace t C $j factors through that of &. □ 

In Subsection 2.3 for arbitrary \i G t* we introduced the left ideals J^' and J^' of the 
algebras A and A respectively. Generalizing (2.14), introduce the quotient vector spaces 

A M , y = A/(j; + I v ) and M M , y = A M)V / J = A/ (J^ + I v + J) (3.1) 

of A. Then A M) y is a left A-module with the space of n-coinvariants M M) y. Note that 
for any element r G X the action of its representative f G G on the algebra A determines 
a linear map 

M M ,y -)■ M T(M)>iF(v) . (3.2) 

Indeed, the adjoint action of f on g preserves the subalgebras n and n'. The action of r 
on the algebra A preserves the right ideal J, and maps the left ideals J^' and \y to J"( M ) 
and I ^(v) respectively. The linear map (3.2) determined in this way is clearly invertible. 
If the weight fx + k is generic, then the spaces of double cosets 

A/(j£' + !v + J) and A/(J , ; + I V + J)=M I1 , V 

are naturally isomorphic. Here we use the equality (2.13). Furthermore, then M M) y is 
isomorphic to the left Z-module, defined as the quotient of the algebra Z by the left 
ideal generated by the cosets of the elements of 1 <E> V and the cosets of the elements 
H — (fx + k)(H) where H G t. Now denote respectively by J" and J" the left ideals of 
A and A , generated by the elements X <g> 1 with iGn'. 

Lemma 3.2. For any element a G & the operator £ CT on A/ J maps 

(J" + T + J)/j -> (J // + l ?(f/ * ) + J)/J. 
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Proof. For each c = 1 , . . . , r denote by n' c the vector subspace of n' spanned by all root 
vectors E a except E c . Denote by J 'J the left ideal of A generated by all elements X <g> 1 
with X <E n' c , and by the element E c . Let J " = U(t) J c be the corresponding left ideal of 
A. We will first prove that the Zhelobenko operator £ c on the vector space A/ J maps 

(J? + J)/J -)• (J"+ J)/J. (3.3) 

The left ideal a c (J") is generated by the element F c , and by the subspace of A formed 
by all elements X <g> 1 where X G n^,. Observe that the latter subspace is preserved by 
the adjoint action of the element F c . Hence for any element Y e A and any element Z 
from that subspace, £ C (YZ) G J"+ J by the definition of the operator £ c . The property 
(3.3) of the operator £ c on A/ J now follows from the inclusion ^ C (YF C ) G J for every 
Y G A. For a detailed proof of that inclusion see [KO, Section 7.2] or [KN1, Section 3]. 

We will now prove Lemma 3.2 by induction on the length £(a) of the element a G & . 
For 1(a) = the statement to prove is tautological. Take any index c G {1 , . . . , r} with 
£(a c a) = £(a) + l. Then £ CTcCT = £ c £a- Note that here we have a~ l (a c ) G A + . Take the 
statement of the lemma as the induction assumption. Observe an equality of left ideals 

J" + Is((7*) = Jc' + I 5(C/*) ( 3 -4) 

of the algebra A. Indeed, by the definitions of left ideals J" and J" the equality (3.4) 
follows from the inclusion 

Kg) C(E C ) G T 5([7 *). 

Using the automorphism cr _1 of the algebra A, the latter inclusion is equivalent to 

l®C(£ a )eI for a = a~ 1 (a c ) e A+, 

which holds by the condition (7) on a Howe system. To make the induction step it now 
suffices to prove that the operator £ c on J\A maps 

(j; + I 5([/ ,)+J)/J (J" + I~ c ~ ([7 , ) + J)/J. 

But that follows from the property (3.3) of the operator £ c , and from the induction step 
we made when proving Lemma 3.1. □ 

Proposition 3.3. Let [i + k G t* be generic. Then the operator £ CT on A/ J maps 

(j; + i + j)/j (j^ + i 5(f/ ,) + j)/j. 

Proof. The left ideal J^' of the algebra A is spanned by V and by all the subspaces 

A(H®1- n(H)) (3.5) 

where H € t. Due to Lemma 3.2, to prove Proposition 3.3 it suffices to consider the 
action of £ CT on the images of the subspaces (3.5) in the quotient J\A. Rewrite (3.5) as 

A(H- (fi + K )(H) -l<g>C(#) + «(#))• (3.6) 
By (1.17) the operator maps the image of Ai7 in J\A to that of A (aoH) . By (2.12) 
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l®C(H)-K(H)el. (3.7) 

Hence by using Lemma 3.1, the image of (3.6) in J\A is mapped by ^ to a subspace of 
the image in J\A of 

k{aoH- (n + «) (#)) + h(u*)- 
We will complete the proof of Proposition 3.3 by showing that for every H G t*, 

<roH-(vL + K )(H)e + h (u *y (3.8) 

Replacing H by a -1 o H at the left hand side of (3.8), we get the element 

H - (a o (p + K ))(H) = H <g> 1 - (a o fi)(H) + 1 <g> ((H) - <t(k)(H) . 

By definition, here 

H®l-(tTop)(H)e 3'^. 

But by applying to the relation (3.7) the automorphism a of A, using the condition (1) 
on a Howe system, and then replacing H by o" _1 (-£f) in the resulting relation, we get 

i®C(h)-<t(k)(h) ei 5([/ .). □ 

Corollary 3.4. Let \i + k G t* be generic. Then for any a G & the operator £ CT defines 
a Z-equivariant linear map 

M M -> M^,^.). (3.9) 

Let us denote this linear map by the same symbol £ CT . Then its Z-equivariance means 

^(^o/)=^(Z)o^(/) for ZeZ and /gM m . 

This property follows from Proposition 1.4 (iii). Note that by (1.16), the above defined 
operator (3.9) maps the weight subspace C M M to the weight subspace 

3.2. Action of Zhelobenko operators 

For any element a G & and any weight /i G t* consider the A -module 

A o- OAt , ?({/*) = A/ (J"o M + )• (3.10) 

Its vector space can be identified with that of the tensor product of the Verma module 
M aoVL over g, multiplied by the H(£/)-module H(U)/H(U)a(U*). The latter quotient 
space can be idenitified with P(a(U)). The vector space of (3.10) is then identified 
with M ff0(l <g) P(a(U)). Further, the space M ffOJli 5m.) of n-coinvariants of (3.10) can 
be identified with P (a(U)) , by assigning to any w <EP (a(U)) the coset of 1 <E> w G A in 

^ao„,a{U*) = A/(J^ 0M + I ?([/ *)+J). (3.11) 

In particular, when a G & is the identity element, the space M M [/* = gets identified 
with P(U) , as in Subsection 2.3. 

Suppose that the weight h + k is generic. Using Corollary 3.4 together with the above 
identifications of the source and target vector spaces in (3.9), the operator ^ on A/j 
determines a linear map P(U) — > P(a(U)) . The latter map will be denoted by I a ,n- 
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Proposition 3.5. Let fx + k G t* be generic. Then for any a G & and any u G P (U) 

I a ^(u) = d(((P <T [p + p})u) (3.12) 
where ((P a [p + p])u is regarded as an element of P (U) by using the definition (2.15). 

Proof. We prove Proposition 3.5 by induction on the length £(a) of a G &. When a is 
the identity element of (5, the statement to prove is trivial. Let us now use the statement 
as the induction assumption. Take any index c G {1, . . . ,r} with £(a c a) = £(a) + 1. 
Then a~ 1 (a c ) is a positive root, let us denote it by a. For short, denote by v the element 
Q{P a [p + p]) u G P (U) . We have 

M CTcCT o M ,a c a(t/*) = A/( J" c(JOAt +la c a(U*) + J)- (3-13) 

Using the definition (1.14) along with the induction assumption, the element I acCF ^(u) 
of P(cr (L 7 ")) can be identified with the coset in (3.13) of the sum 

oo s 

J2 II {HHc-t + l))- 1 -E* ad 8 Fc (l® a c a(v)). 

s=0 t=l 

Without changing the sum, we can replace the operator ad^ c by l<g>ad£(_p c ) here, because 
the elements F C <S> 1 and 1 ® a c a(v) of A commute. Without changing the coset, we can 
then replace E£ by 1 <g> ((E£) in resulting sum, because E c <g> 1 commutes with 

1 <g> &d s c{Fc) (a c a(v)) 

and belongs to the left ideal J" C J" c<TOM - So we get the sum 

oo s 

^ J] (t^e-t+l^-'-l^C^ad^)^^)) = 

s=0 t=l 

oo s 

£ II (t(^c-t + l))" 1 -l®a c a(Craad^ a) (z;)). 

s=0 t=l 

By the property (7) of a Howe system, the element 1 ® C(-^a) belongs to the left ideal 
I = I (7* of A . Hence 

1 <g> a c a(C(^a)) £ 

Therefore in the last displayed sum, the element a,d s ^ Ea ^(v) G H({7) can be replaced by 
the element C(-^a) u e H(LZ) , without changing the coset of the sum in (3.13). We get 

oo s 

nW^c-i+l))" 1 -l®5c5(C(0««). (3-14) 

s=0 t=l 

Here -£f c stands for H c <g) 1 + 1 <8> ((H c ) . Modulo the left ideal J" c(JOAt of A, the element 
i? c ® 1 equals 

(cr c cr o fu)(H c ) = (o- c O-(p + p) - p)(H c ) = 

(p + P ){{a c a)-\H C )) - p{H c ) = -(/* + p) - 1 . 
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Hence the coset of the sum (3.14) in (3.13) coincides with that of 

oo s 

E n i®(HaH c )-^+ P )(H ol )-t))- 1 -i®s c z(aF:)aK)v) = 

s=0 i=l 

oo s 

E (t(-C(H a )-^ + p)(H a )-t))- 1 -C(F^C(E^v) = 

s=0 t=l 

i®d c d{C{p a [n + p))v) = i®a c d{c{p ac(T [iJL + P ])u). □ 

The operator (3.9) has been defined only when the weight p + k is generic. However, 
Proposition 3.5 yields the following result, which is valid for any p G t*. Here we also 
use the remark made at the very end of Subsection 3.1. 

Corollary 3.6. If A + p is nonsingular, the operator ( a on A/ J defines a linear map 

K -> Mni, 9( ^). (3-15) 

Proof. The source and target spaces in (3.9) can be identified with P(U) and P(a(U)) 
respectively. The first of these identifications uses the bijection t M as in Subsection 2.3. 
Let u G P (U) be any element of weight A — p relative to the action of t. Here any element 
H E t acts on P (U) via the left multiplication by ((H) , using the definition (2.15). Then 
tfj,(u) G M*. When applying the operator ((P a [p + p]) to u, the denominators become, 
up to non-zero scalar multipliers, certain products of the factors of the form 

(A - p)(H a ) + (p + p)(H a ) + s = (A + p)(H a ) + s 

where a is a positive root and s is a positive integer. These factors do not depend on p,, 
and do not vanish if the weight A + p is nonsingular. Thus by mapping u to the element 
of P(a(U)) at the right hand side of (3.12), we get the required linear map (3.15). □ 

3.3. Irreducibility theorem 

Now take the longest element o~o of the Weyl group & . For any weight p & t* consider 
the R-module M ao0 ^^ ^jj*^ . The next Proposition 3.7 establishes a connection between 
the Shapovalov form on , and the linear map 

M a o/j,,a {U *) V / 

defined by the Zhelobenko operator £o = (<j on A/ J, when A + p G i* is nonsingular. 
The latter map will be denoted by £o I . Note that when p + K G t* fails to be generic, 
the operator £ on A / J does not necessarily define any map from the whole space . 
By combining Proposition 3.7 with Corollary 2.7, we will obtain Theorem 3.8. 

Proposition 3.7. Let A + p G t* be nonsingular. Then for any p G t* 

KerSj = Ker(&|M£). 
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Proof. According to Subsection 3.2, the vector space of the the R-module M^o^ t $ m*) 
can be identified with P (do(U)) . Denote by ( , ) ao the non-degenerate C-bilinear pairing 

P{U) x P(a (U))^C 

defined by the equality 

(u,w) ao = (u,a^ 1 (w)}. (3.17) 

Here ( , ) is the non-degenerate symmetric bilinear form on the vector space U selected 
in Subsection 2.1. Hence the pairing ( , ) ao is non-degenerate too. 

Now for any «,«6P (U) put / = t^,(u) and g = i^(v) . If the weight p + k is generic, 
then by Propositions 2.3 and 3.5 we have 

Si(f,g) = (u,((P[» + p])v) = (u,a -\l (To ^))) = (u,I ao ,M) ao . (3.18) 

When the difference A — p and the elements u,v G P (U) are fixed, the left hand side 
of the above equalities becomes a rational functions of p G t* . If A + p is nonsingular, 
this rational function has finite values for all p by Proposition 2.4. Now Proposition 3.7 
follows from the definition of the operator £ I , see our proof of Corollary 3.6. □ 

Theorem 3.8. Suppose that A + p G t* is nonsingular, and that D\\ = &\ . Then for 
any p G t* the quotient / Ker(£ I ) is an irreducible Q -module. 

Recall that the algebra A is specified as the tensor product (2.5), where the group 
G acts diagonally. Consider the corresponding subalgebra A G C A of G -invariants. 
This subalgebra acts on A ^ by restricting the action of A. Since A G C Norm J, the 
subalgebra A G then acts on the space of n-coinvariants of A^. The latter action 
preserves the subspace C M M . This subspace is also a Q -module, if the weight A + p 
is nonsingular. The above action of A G on can also be obtained by pulling the action 
of Q back through the isomorphism 7 : A G — > Q, see Subsection 1.5. Note that for any 
a £ & the actions of A G on the source and target vector spaces of the map (3.15) 
defined by are intertwined by this map by its definition. Using this observation 
when a = o"o, we obtain a corollary to Theorem 3.8. 

Corollary 3.9. Suppose that A + p G t* is nonsingular, and that 9\\ = &\ . Then for 
any p, G t* the quotient / Ker(£ I ) is an irreducible A G -module. 



3.4- Contravariant pairing 

For any p G t* we can define a non-degenerate C-bilinear pairing 

as follows. For any two elements u G P (U) and w G P(ao(U)) consider the cosets in M M 
and M^ojj ; ao(u* ) of 1 and 1 <g>w respectively; see (3.11). By definition, the value of 
Qfj, on this pair of cosets is (u,w) ao ; see (3.17). By restricting to weight subspaces, 
we define a pairing 

: x M cr ° oA ~ ->■ C (3.19) 

™ M M a Ofi ,a (U*) V / 
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for any A G t* . The latter pairing is also non-degenerate, see the end of Subsection 2.3. 
Now suppose that the weight A + p is nonsingular. Let us prove that then 

Q^(e(Z)of,h) = Q^(f,Uz)oh) when ZeS. (3.20) 

In particular, here £o(Z) o h G M^°°^ $ (u*) * s defined, even though the element £o(Z) 
of the algebra Z may be not contained in the subspace Z C Z . 

First let p, + k be generic. Then the map (3.16) defined by the Zhelobenko operator 
£o on A/ J, is invertible. Let g G be the image of h under the inverse map. By (3.18) 

Q^e(Z)of,h) = S^e(Z)of,g) = S^f,Zog)=Q^f,UZ)oh). 

Here we used (2.30) and Corollary 3.4. Thus we get (3.20) for generic p + k. Now note 
that when A + p is nonsingular, the left hand side of (3.20) is defined for any weight p, 
as a finite number. When /, h, Z and the difference A — p are all fixed while p varies, 
this number becomes a rational function of p. Hence the right hand side of (3.20) can 
also be defined as a rational function of the weight p, with only finite values. Since the 
pairing (3.19) is non-degenerate, we can then determine the vector £o(Z) ° h. 

If Z G Q, then io(Z) = Z by definition. Hence the equality (3.20) implies that for 
any p and nonsingular A + p the pairing (3.19) is Q -contravariant : 

Q^(e(Z)of,h) = Q^(f,Zoh) when ZeQ. 
In particular, we have an action o of the algebra Q on the vector space M^°°^ a (u*) • 



4. Yangians and reductive dual pairs 

4-1. Reductive dual pairs 

From now on we will work with examples of Howe systems, as defined in Subsection 2.1. 
For each of the corresponding algebras (2.5) we will describe explicitly its subalgebra of 
G-invariants. Then we will apply Corollary 3.9 to each of these examples. The group G 
will be one of the classical complex Lie groups GL m , Sp 2m and 02 m with any positive 
integer m. Respectively, g will be one of the Lie algebras rj[ m , sp 2m and S02 m . 

First consider G = GL m . Let the indices a and b run through 1 , . . . , m . Then e a will 
denote a vector of the standard basis in the vector space C m , while E a b G gl m will be 
a standard matrix unit. Choose the standard triangular decomposition (1.1) of g = gl m 
where the subalgebras n,t,n' are spanned by the matrix units E a i with a > 6, a = 6, 
a < b respectively. The elements E aa form a basis in t, and we will denote by rj a the 
vector of the dual basis in t* corresponding to E aa . The positive and negative roots are 
Va — Vb with a < b and a > b respectively. The semisimple rank r of rj[ m is m — 1 . For 
each c=l,...,m — lwe will choose r] c — r] c+ i as the simple root a c , and 

He = E cc — E c+ i^ c+ i , E c = E CjC+ i and F c = E c+ i jC 

as the basis elements of the s [2 -subalgebra g c C g. Define the Chevalley anti-involution 
e on g[ m by setting e(E a t,) = Eb a ■ Choose the trivial Cartan decomposition g = g + Q)g~ 
of gl m so that g + = gl m and rj_ = {0}. 
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The algebraic group GL m is connected, and the maximal torus T C GL m with the 
Lie algebra t consists of all elements acting on each basis vector e a G C m by a scalar 
multiplication. The action of a c G 6 on t* exchanges the dual basis vectors 7] c and r] c+ i , 
leaving all other basis vectors fixed. Hence the Weyl group & can be identified with the 
symmetric group & m . Choose the representative a c G NormT so that its action on C m 
exchanges the basis vectors e c and e c+ i , leaving all other basis vectors of C m fixed. 
Note that here the group X is trivial. 

Now consider G = Sp 2m , • Let a and b run through — m, . . . , — 1 , 1 , . . . , m. Then 
e a will denote a vector of the standard basis in C 2m , and E ab will be a standard matrix 
unit in jj^m- We will regard G as the subgroup in GL2 m preserving the bilinear form 
on C 2m whose value on any pair (e a ,e b ) of the basis vectors is respectively 

<y o _6-signa or <f _ 6 

when G is Sp 2m or 02 m . Then g is the Lie subalgebra of g( 2m spanned by the elements 

F ab = E ab - sign ab ■ E- b - a or F ab = E ab - E_ b _ . 

Choose the triangular decomposition (1.1) of g = 5p 2m ,so 2m where the subalgebras 
n,t,n' are spanned by the elements F ab with a > b, a = b, a < b respectively. The 
elements F_ a _ a with a > form a basis in t. Here for any a > we will denote by i] a 
the vector of the dual basis in t* corresponding to the basis vector F a _ m _i ja _ m _i G t. 
The positive roots of sp 2m are rj a — rj b and i] a + i] b where 1 ^ a < b ^ m, together with 
2rj a where 1 ^ a ^ m. The positive roots of S02 m are only rj a — t] b and r\ a + i] b where 
1 ^ a < b ^ m. The semisimple rank r of so 2m with m > 1 and of sp 2m is m. But the 
the semisimple rank of SO2 is zero, while the root system is empty. 

For g = sp 2m ,S02m and c = 1 , . . . , m — 1 choose a c = r\ c — t] c+ i and 

H — F 1 1 — F E = F 1 and F = F 1 

Further, if g = sp 2m then choose a m = 2rj m and 

H m = F_i ; _i , £' m = F_i ; i/2 and F m = Fi ; _i/2. 

If g = S02m and m > 1 , then choose ct m = ?7 m _i + ?] m and 

H m = F-2,-2 + , E m = F-2,i and F m = Fi ; _ 2 . 

For g = sp 2m ,£02m choose the Cartan decomposition g = g + ©g_ where g + is spanned 
by the elements F ab with a, 6 > while g_ is spanned by F a> _ b and -F_ a ^ with a, 6 > 0. 
The Chevalley anti-involution e will be defined by setting s(F ab ) = signal -F ba if 9 = 1 , 
or £(-F a b) = -F6 a if 9 = — 1. Here the parameter 9 is the same as in Subsection 2.1. This 
choice of s is prescribed by the condition (3) on our particular Howe systems, see below. 

The algebraic group Sp 2m is connected, but 02 m has two connected components. 
For G = Sp 2m ,02 m the maximal torus T C Go with the Lie algebra t consists of all 
the elements of GL2 m which multiply any two basis vectors e a ,e_ a G C 2m by scalars 
inverse to each other. The Weyl group of sp 2m is isomorphic to the semidirect product 
& m k where the symmetric group & m acts by permutations of the m copies of Z2 . 
The Weyl group of so 2m is isomorphic to a subgroup of 6 m x of index two. For 
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c = 1 , . . . , m — 1 and g = sp 2m ,50 2m the action of a c G & on t* exchanges the basis 
vectors r\ c and rj c+ i , leaving other basis vectors fixed. Then choose the representative 
a c G Norm T so that its action on C 2m exchanges e c _ m _i and e c _ m , also exchanges 
e m - c+ i and e m _ c , leaving all other basis vectors of C 2m fixed. For g = sp 2rn we have 
o~m(Vm) = — Tim and o~m(Va) = Va for 1 ^ a < m . Choose the representative a m G Sp 2m 
so that (7 m (e_i) = e\ and cr m (ei) = — e_i while cr m (e a ) = e a for |a| > 1. Note that for 

= sp2m the group X is trivial, like it was in the case g = gl m . 

Now take g = 50 2m . Here the group 1 is not trivial, but is isomorphic to Z 2 . Let r m 
be the generator of this group. Then r m (r] m ) = —rj m and r m (n a ) = n a for 1 ^ a < m, 
so that a m = Tmam-iTm. Choose the representative r m G 2m so that r m (e_i) = ei 
and r m (ei) = e_i while r m (e a ) = e a for |a| > 1. Choose a m G 02 m to be r m a m _i r m . 
We will need a representative in 02m for every element w G 91 . If u; ^ (5 , then w = r m a 
for some a G (3 . In this case, the representative of w in 02 m will be Q = r m a . 

We will now associate to any G another classical complex Lie group, to be denoted 
by G'. Let n be any positive integer. If 9 = 1 then for G = GL m ,Sp 2m ,0 2m put 
G' = GL n ,O n ,Sp n respectively. If 9 = —1 then put G' = GL n ,Sp n ,O n respectively. 
Here for G' = Sp n the integer n is to be even. Then (G, G') is a reductive dual pair [HI]. 

Let g' be the Lie algebra of G'. Let the indices i and j run through the sequence 
1 , . . . , n . Then /j will denote a vector of the standard basis in the vector space C n , 
while Eij G gl n will be a standard matrix unit. If i is even, put i = i — 1 . If i is odd and 

1 < n, put % = i + 1 . Finally, if i = n and n is odd, put i = i. We will regard G' = O n or 
G' = Sp n as the subgroup in GL n , preserving the bilinear form on C n whose value on 
any pair (fi,fj) of the basis vectors is 0* 5^- where 9i = 1 or 9i = (— l) l_1 respectively. 
Then g' is the Lie subalgebra of gl n spanned by the elements — 9i9jE^ . 

Let U be the tensor product of vector spaces C m ® C n , and let U* be the dual vector 
space. For a = 1 , . . . , m and % = 1 , . . . , n let x a i denote the basis vector e a <E> fi of U . 
Then let d ai denote the corresponding vector of the dual basis in U* . If 9 = 1 then 
P(U) is the algebra of polynomials in x ai while H(L r ) can be identified with the algebra 
of differential operators on P(U) , so that d a i is the partial derivation corresponding to 
x a i. If 9 = — 1 then P{U) is the Grassmann algebra with mn anticommuting generators 
x a i. The definition (2.15) then implies that the element x a i G H(?7) acts on P(U) via 
left multiplication, while d ai G H(C7) acts as the left derivation relative to x a i- The 
latter operator on the Grassmann algebra is also called the inner multiplication by x a i . 
For any 9 the form ( , ) on U will be chosen so that the basis of the x a i is orthonormal. 
Then the involutive anti-automorphism e of the algebra H (U) exchanges x a i with d a i . 

Let us now consider the vector space W = U © U* . The groups GL m and GL n act on 
the vector space U, and their actions commute which other. Hence we get the mutually 
commuting actions of GL m and GL n on W . Clearly, both actions preserve the bilinear 
form B in W as introduced in Subsection 2.1. If G = Sp 2m ,02 m then we can identify 
the vector space W with the tensor product C 2m ® C n so that for a = 1 , . . . , m we 
have x a i = e. a ® fi as above, and d a i = e_ a <8> (9i fa) . Then the bilinear form B on W 
gets identified with the tensor product of the forms on C 2m and C n chosen above. If 
9 = 1 then the form B is alternating and G' = O n , Sp n respectively. If 9 = —1 then the 
form B is symmetric and G' = Sp n ,O n respectively. Hence we always obtain mutually 
commuting actions of G and G' on the vector space W , preserving the bilinear form B . 
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To complete the description of a Howe system on U one requires a homomorphism 
C : U(g) — > H(£7) obeying the conditions (1) to (6) from Subsection 2.1. It is well known 
and can be verified directly that C can be chosen so that for g = gt m and a, b = 1 , . . . , m 

n 

C(E ab ) = x aidbi-, (4.1) 
1=1 

while for g = sp 2m ,S02 m and the same indices a, 6 = 1 , . . . , m 



((F ab ) = 65 ab n/2 + ^2x a id b i, 

i=i 

n n 

C(F a ,_ 6 ) = -J2 0e iXa ~ lXbi , C(F-a,b) = 9 i d aidm- (4.2) 



i=i i=i 



For g = gl m ,sp 2m ,S02m and 6 = 1, — 1 by [H2, Sections 2.3, 3.5, 3.8,4.2, 4.3] the image of 
the homomorphism C coincides with the subalgebra of G'-invariant elements in H ([/"). 
Note that for any i , j = 1 , . . . , n the element Eij e gl n acts on P(U) as the operator 



x ™ d aj ■ (4.3) 

a=l 

Hence for g' = so n ,5p n the element — OiOjE^ £ g' acts on P(U) as the operator 

m 

Y ( ^ai d aj - 6i 9j x a5 d a i ) . (4.4) 



a=l 



It is well known [HI] that the subalgebra of G-invariant elements in H(£/") is generated 
by the elements (4.3) for G = GL m or by the elements (4.4) for G = Sp 2m ,02 m . In 
Subsection 4.3 we give an analogue of this result for the algebra (2.5) instead of H({7). 

4-2. Yangians 

First take the Yangian Y(gt n ) corresponding to the Lie algebra g' = gl n . This Yangian 

is a complex unital associative algebra with a family of generators T^\t^ , . . . where 
i,j = 1 , . . . , n . Defining relations for these generators can be written using the series 

T lJ (x) = 5 lJ +T^x- 1 +T^x- 2 + ... 

where a; is a formal parameter. Let y be another formal parameter. Then the defining 
relations in the associative algebra Y(gt n ) can be written as 

(x-y)[T i:i (x),T kl (y)] = T kj {x)T a (y) - T kj {y)T u (x) . (4.5) 

The algebra Y(gt n ) is commutative if n = 1 . By (4.5), for any zGC the assignments 

Tij(x) ^ T i:i (x + z) (4.6) 
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define an automorphism of the algebra Y(gl n ) . Here each of the formal power series 
Tij(x + z) in (x + z)~ l should be re-expanded in a; -1 , and every assignment (4.6) is a 
correspondence between the respective coefficients of series in x~ x . Relations (4.5) also 
show that for any formal power series g(x) in x~ x with coefficients from C and leading 
term 1, the assignments 

Tij{x) ^ g{x)Tij{x) (4.7) 

define an automorphism of the algebra Y(gl n ). The subalgebra in Y(gl n ) consisting of 
all elements which are invariant under every automorphism of the form (4.7), is called 
the special Yangian of g\ n , and will be denoted by SY(rj[ n ) . Two representations of the 
algebra Y(gl n ) are called similar if they differ by an automorphism of the form (4.7). 
Similar representations of Y(gl n ) have the same restriction to the subalgebra SY(g[ n ) . 
Using (4.5), one can directly check that the assignments 

Tij(x) ^ S id +E ij x~ 1 (4.8) 

define a homomorphism of unital associative algebras Y(gl n ) — > U(gl n ) . There is also 
an embedding XJ(gl n ) — > Y(gl n ) , defined by mapping Eij I— > tA 1} . So Y(g[ n ) contains 
the universal enveloping algebra U(gl n ) as a subalgebra. The homomorphism (4.8) is 
identical on the subalgebra U(gl n ) C Y(gl n ) . 

Let T(x) be the nxn matrix whose i,j entry is the series Tij(x) . One can show that 
the assignment 

T{x) ^ T(-x)- 1 (4.9) 

defines an involutive automorphism of the algebra Y(gl n ) . Here each entry of the inverse 
matrix T(— x)~ l is a formal power series in x~ x with coefficients from the algebra Y(gl n ) , 
and the assignment (4.9) as a correspondence between the respective matrix entries. 

The Yangian Y(gl n ) is a Hopf algebra over the complex field C. The co multiplication 
A : Y(gl n ) — > Y(gl n ) <8> Y(g[ n ) is defined by the assignment 

n 

A : T^x) ^ T ^ x ) ® T ^ x ) ■ ( 4 - 10 ) 
k=i 

When taking tensor products of Y(g( n ) -modules, we use the comultiplication (4.10). 
The counit homomorphism Y(g( n ) — > C is defined by the assignment Ty(x) h- > 5ij . The 
antipodal map Y(gl n ) — > Y(gl n ) is defined by mapping T(x) i — y T(x)- 1 . Note that the 
assignments 

Tijitf^Tjiix) (4.11) 

define an involutive anti-automorphism of the associative algebra Y(gLj. Moreover, 
they define a bialgebra anti-automorphism of Y(gLj . 

Note that the special Yangian SY(gLj is a Hopf subalgebra of Y(g( n ) . It is isomorphic 
to the Yangian Y(sl n ) of the special linear Lie algebra sl n C gl n considered in [Dl, D2]. 
For the proofs of the latter two assertions see [M2, Subsection 1.8]. 

Now let g' be one of the two Lie algebras so n ,sp n . When considering these two cases 
simultaneously, we will use the following convention. Whenever the double sign ± or 
=F appears, the upper sign will correspond to the case of a symmetric form on C™ so 
that g' = so n , while the lower sign will corresponds to the case of an alternating form 
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on C n so that g' = sp n . Let T(x) be the transpose to the matrix T(x) relative to that 
form on C n . The i,j entry of the matrix T(x) is 9i9jTj^(x) , see Subsection 4.1. An 
involutive automorphism of the algebra Y(gl n ) can be then defined by the assignment 

T(x)^f(-x). (4.12) 

This assignment is understood as a correspondence between respective matrix entries. 
Note that (4.12) defines an anti-automorphism of the coalgebra Y(gl n ) , like (4.11) does. 
Consider the product T{— x) T(x) . The i,j entry of this matrix is the series 

n 

J20i9 k T- k -(-x)T kj (x). (4.13) 
fe=i 

The twisted Yangian corresponding to the Lie algebra g' is the subalgebra of Y(g( n ) 
generated by coefficients of all series (4.13). We denote this subalgebra by Y(g'). The 
subalgebra Y(g') n SY(gl n ) of Y(g( n ) is called the special twisted Yangian corresponding 
to g' . This subalgebra will be denoted by SY(g'). The automorphism (4.7) of Y(gl n ) 
determines an automorphism of Y(g') which multiplies the series (4.13) by g(x) g(—x) . 
The subalgebra SY(g') of Y(g') consists of the elements fixed by all such automorphisms. 
Two representations of Y(g') are called similar if they differ by such an automorphism. 
Similar representations of Y(g') have the same restriction to the subalgebra SY(g'). 

To give defining relations for these generators of Y(g') , let us introduce the extended 
twisted Yangian X(g'). The complex unital associative algebra X(g') has a family of 
generators S- 2 \ . . . where i ,j = 1 , . . . , n . Put 

^■(x) = ^ + 4 1) x- 1 + 4 2) x- 2 + ... 

and let S(x) be the nx n matrix whose i,j entry is the series Sij(x) . Defining relations 
in the algebra X(g') can then be written as 

(x 2 - y 2 ) [Sij(x), S k i(y)] = (x + y)(S k j(x) Su(y) - S k j(y) S u (x)) 
T(x-y)( e k 6 3 S i% {x) S~ 3l {y) - 4 0« S k M S u {x)) 

± etBjiSkiix) Sji(y) - S ki (y) S 5l (x)) . (4.14) 

These relations shows that for any formal power series f(x) in x~ l with the coefficients 
from C and leading term 1, an automorphism of the algebra X(g') is defined by mapping 

Sij{x) H- f(x)S ij (x). (4.15) 

Let S(x) be the transpose to the matrix S(x) relative to our form on C", so that the 
i,j entry of S(x) is 6i6jSjj(x) . By [M2, Theorem 2.3.13] there is a formal power series 
O(x) in x~ l with the coefficients in the centre of X(g') and leading term 1, such that 

S{x) =F 2xS(x) = (1 =f 2x) O(x) S(-x) . (4.16) 

Moreover, then 0{x)0{—x) = 1. Note that (4.16) yields an explicit formula for 0(x). 
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One can define a homomorphism X(g') — > Y(g') by mapping 

S(x) ^ f(-x)T(x). (4.17) 

Moreover, the kernel of the homomorphism (4.17) is generated by the coefficients of the 
series O(x) — 1. For the proof of the last two statements see [M2, Section 2.13]. By 
setting 0(x) = 1 in the matrix relation (4.16) and then considering the i,j entry we get 

S ij {x)^2x9 i 9 j S ri {x) = (1t2x)S ij (-x). (4.18) 

The homomorphism (4.17) is surjective by its definition. Thus the twisted Yangian Y(rj') 

can also be defined as the associative unital algebra with the generators . . . 

subject to the relations (4.14) and (4.18). 

Note that the anti-automorphism (4.11) and the automorphism (4.12) of the algebra 
Y(g[ n ) commute with each other. Their composition is an involutive anti-automorphism 
of Y(gl n ) which maps 

T i:j {x) ^ 6i6jT- t - 3 {-x). (4.19) 

The composition (4.19) preserves the the subalgebra Y(g') C Y(g[ n ). The resulting 
anti-automorphism of Y(g') can also be obtained as follows. By using (4.14) only, one 
shows that the assignments 

Sij(x) ^ Sji(x) (4.20) 

define an involutive anti-automorphism of the algebra X(g') ; see [M2, Proposition 2.3.4]. 
Moreover, due to (4.18) it factors to anti-automorphism of the algebra Y(g') . The latter 
coincides with the restriction of (4.19) to the subalgebra Y(g') C Y(gl n ) . 

Suppose we are given a representation of the algebra X(g') such that every coefficient 
of the series 0(x) is represented by a scalar. Denote by o(x) the corresponding series 
with scalar coefficients. Then o(x) o(—x) = 1 and the leading term of o(x) is 1. Hence 
we can find another formal power series f(x) in x~ l with scalar coefficients and the 
leading term 1 , such that o(x) = f(—x)/f(x) . By pulling back the given representation 
of X(g') through the automorphism (4.15) we then get another representation of X(g') , 
which factors through the homomorphism X(g') — > Y(g') . The series f(x) and hence the 
resulting representation of Y(g') are not unique. However, here we can only replace f(x) 
by f(x) h(x) where h(x) is a formal power series in x~ l with scalar coefficients and the 
leading term 1 , such that h(x) = h(—x) . Then h(x) = g(x) g(—x) for some series g(x) as 
in (4.7). Hence all the representations of Y(g') corresponding to the given representation 
of X(g') are similar to each other, and have the same restriction to SY(g') . 

The twisted Yangian Y(g') has an analogue of the homomorphism Y(gl n ) — > U(g( n ) 
defined by (4.8). Namely, one can define a homomorphism X(g') — > U(g') by mapping 

S ij (x)^6 ij + Eij - 9 < d > E >* (4.21) 

X ± 2 

This can be proved by using the defining relations (4.14), see [M2, Proposition 2.1.2]. 
Moreover, the homomorphism (4.21) factors through the homomorphism X(g') — > Y(g') 
defined by (4.17). Further, there is an embedding U(g') — > Y(g') defined by mapping 
each element Eij— OiOjEji e g' to the coefficient atx -1 ofthe series (4.13). Hence Y(g') 
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contains the universal enveloping algebra U(g') as a subalgebra. The homomorphism 
Y(g') — > U(g') corresponding to (4.21) is identical on the subalgebra U(g') C Y(g') . 

The twisted Yangian Y(g') is not only a subalgebra of Y(gl n ) , it is also a right coideal 
of the coalgebra Y(gl n ) relative to the comultiplication (4.10). Indeed, let us apply this 
comultiplication to the i,j entry of the n x n matrix T(—x) T{x) . We get the sum 

n 

J20i0kA(T~ k ~ t (-x)T kj (x)) = 
fc=i 

n 

E d i d A T kg(- X )® T ~9i(-x))( T kh(x)®T hj (x)) = 
g,h,k=l 

n 

Yl 0g0kT- k§ (-x)T kh (x) ® 6i6 g T § - t (-x)T hj (x). 

g,h,k=l 

In the last displayed line, by performing the summation over k = 1 , . . . , n in the first 
tensor factor we get the g,h entry of the matrix T(—x) T(x) . Therefore 

4(Y(B'))cY( fl ')«Y( fl g. (4.22) 

For the extended twisted Yangian X(g') , one defines a homomorphism of associative 
algebras 

X(g / )^X(g')®Y(gl n ) 

by mapping 

n 

Sij(x) ^ S gh (x)®0i0 g T § - t (-x)T hj (x). (4.23) 

g,h=l 

The homomorphism property can be verified directly, see [KN3, Section 3]. Using the 
homomorphism (4.23), the tensor product of any modules over the algebras X(g') and 
Y(gl n ) becomes another module over X(g'). 

Moreover, the homomorphism (4.23) is a right coaction of the Hopf algebra Y(gl n ) 
on the algebra X(g') . Formally, one can define a homomorphism of associative algebras 

X(g')^X(g / )®Y(gl n )®Y(gy 

in two different ways: either by using the assignment (4.23) twice, or by using (4.23) 
and then (4.10). Both ways however lead to the same result, see again [KN3, Section 3]. 

4-3. Olshanski homomorphisms 

First consider the reductive dual pair (G,C) = (GL m ,GL n ). We will treat the cases 
of 6 = 1 and 6 = — 1 simultaneously. We shall use the Yangian Y(gl n ) to describe the 
subalgebra of GL m -invariant elements in the corresponding algebra (2.5). Denote by E 
the mxm matrix whose a, b entry is E^ a e gt m . Note the transposition of the indices a 
and b here. The inverse matrix (x + 9E) _1 will be regarded as a formal power series in 
x~ x whose coefficients are certain mxm matrices with entries from the algebra U(gl m ) . 
Let (x + 0E)~^ be the a, b entry of the inverse matrix. This entry equals 
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oo m 

5 ah x~ l -9E ha x- 2 + J2 Yl (~ d ) S+1 E Cia E C2Cl . . . Ecc^Ebc, x~ s ~ 2 . 

s=0 d,...,c s — l 

The following result first appeared in [01], although in another guise. For connections to 
our setting see [KN1, Section 4] or [KN2, Section 4] when 9 = 1 or 9 = — 1 respectively. 
Let U(rj( m ) GLm be the subalgebra of GL m -invariants in U(rj( m ) ; it coincides with the 
centre of the algebra U(gl m ) . 

Proposition 4.1. (i) One can define a homomorphism Y(gl n ) — > XJ(gl m ) ® H(C/) by 

m 

T i:i (x) ^ Sij + (x + 9m/2 + 9E)-^x ai d bj . (4.24) 

a,b=l 

(ii) The subalgebra of GL m -invariant elements in A = U(rj( m ) <S> H(CT') coincides with 
the subalgebra, generated by U(f(l m ) GLm ®1 and by the image of homomorphism (4.24). 

Proof. Part (i) of the proposition was proved in [KN1, Section 1] and [KN2, Section 1] 
for 9 — 1 and 9 — — 1 respectively. The claim that the image of the homomorphism (4.24) 
belongs to the subalgebra A GLm C A , was also proved therein. It remains to prove that 
the elements 

m 

l®x ai 3 aj (4.25) 

a=l 

and 

^ E Cl aE C2Cl . . . E C3C3 _ 1 Ebc 3 <S> x a i dbj , (4.26) 

o,6,ci,...,c s =l 

together with the elements of U(rj( m ) GLm ® 1 , generate the whole subalgebra A GLm C A . 
Here s = , 1 , 2 , . . . and i,j = 1 , . . . , n . For s = the first tensor factor of the summand 
in (4.26) should be understood as E\, a . 

Let N denote the additive semigroup of non-negative integers. Take the standard 
N-filtration on the algebra U(rj[ m ) , where any element of rj[ m has degree 1 . The adjoint 
action of the group GL m on U(gl m ) preserves this filtration, and the corresponding 
graded algebra is identified with the symmetric algebra S(gl m ) . The algebra H(L r ) has 
an N x N-filtration, such that the elements of U and U* have degrees (1,0) and (0,1) 
respectively. This filtration is preserved by the action of the group GL m on H (U) . The 
corresponding graded algebra is identified with the symmetric algebra of U © U* in the 
case 9 = 1 , or with the exterior algebra of U © U* in the case 9 = — 1 . In both cases, the 
graded algebra is denoted by P(U © U*). It suffices to prove that the elements of the 
algebra S(rj( m ) ® P(U © U*) corresponding to (4.25) and (4.26), taken together with 

S(flU GLm ®l C S( Lj®P(£/©tn, (4.27) 

generate the whole subalgebra of GL m -invariants. We will prove this for n = 1 only. 
The generalization of our proof to any n ^ 1 will be obvious; cf. [MO, Subsection 2.9]. 

So let us suppose that U = C m <E> C 1 = C m . Let e[ , . . . , e' m be the basis of the vector 
space U* dual to the standard basis e\ , . . . , e m of U . First consider the GL m -invariants 
in the subspace 
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1®P(U®U*) C S(gl m )®P(U ®U*). (4.28) 

The subspace of P ( U (BU*) of degree (p , q) G N x N is a subspace in the tensor product 
eg) (£/"*) ® g . The latter tensor product contains non-zero GL m -invariant vectors, only 
if p = q. The GL m -invariants in JJ® P <8> (£7"*)® p are any linear combinations of the sums 

m 

e c ro(1) ® ■ ■ ■ ® e Cro(p) <g> < <g> . . . <g> (4.29) 

Cl , . . . ,Cp — 1 

where w ranges over all permutations of the indices 1 , . . . , p. Here we used the classical 
invariant theory for GL m ; see [HI, Theorem 1A]. For 9 = 1 or 9 = — 1 respectively, by 
applying to the sum (4.29) the symmetrization or antisymmetrization in the first p and 
also in the last p tensor factors of the summand, we get an element of 

P(U)®P(U*) =P(U®U*) 

corresponding to 

m 

Y x Cll ...x Cpl d Cll ...d Cpl eR(U) 

multiplied by 9 l ^ . This observation shows that for n = 1 , the GL m -invariants in the 
subspace (4.28) are generated by the elements corresponding to (4.25) with i = j = 1. 

Now consider the subspace (4.27). Generators of the subalgebra S(gl m ) GLm C S(gl m ) 
are well known. We will reproduce a set of generators here, because they will be used in 
the subsequent argument. Identify gl m with U <8> U* as a module of the group GL m , so 
that the matrix unit E a f, e Q\ m is identified with the vector e a ®e' b G U ®U* . Then the 
subspace in S(gl m ) of degree t becomes a subspace in (U (8) U*)® 1 . The GL m -invariants 
in the latter tensor product are any linear combinations of the sums 

m 

ec -(D e 'ci ® • • • ® e ^ ( «) ® < (4-30) 

a,...,c t =l 

where ct7 ranges over all permutations of the indices 1 , . . . , t . Let si , . . . , be the cycle 
lengths of the permutation w , so that t = s± + . . . + Sk - By applying to the sum (4.30) 
the symmetrization in the t pairs of tensor factors of the summand, we get an element 
of S(gl m ) , which corresponds to the product over s = si , . . . , Sk of the elements 

m 

E C2C1 E C3C2 . . . E CaCa _ x E ClCa G U( [ m ) GL - . (4.31) 

Cl,...,C s = l 
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Now take the subspace in S(gl m ) ®P (U ©£/*) of degree t + 1 in the first tensor factor 
and of degree (p, q) in the second tensor factor, for any t ^ and (p,q) 7^ (0,0) . Regard 
it as a subspace in the tensor product 

(U® U*)® (t+1) <g> U® p <g> (U*)® q . 

The tensor product contains non-zero GL m -invariant vectors, only if p = q. Suppose this 
is the case. Then the GL m -invariants in the tensor product are any linear combinations 
of the sums over the indices cq , c\ , . . . , ct , ct+i , . . . , ct+ p = 1 , . . . , m of the vectors 

e Cro(0 ) ®e' CQ ®... ®e Cza{t) <g> e' Ct <g> e Cro(t+1) <g> . . . <g> e Cro(t+p) <g> e£ t+1 <g> . . . <g> e^ t+p (4.32) 

where each of the sums corresponds to a permutation zuofO,l,...,t,t+l,...,t + p. 
The GL m -invariants in S(gl m ) ® P(L7" © f7*) are obtained by applying to these linear 
combinations the symmetrization of the first t + 1 pairs of factors of the tensor product 
(4.32), and the symmetrization or antisymmetrization in the next p and in the last p 
factors, for 9 = 1 or 9 = — 1 respectively. Let Q ro be the element of S(rj( m ) <g>P({7 ©£/*) 
obtained by these operations from the sum corresponding to the permutation w . 

If the cycle of an element of the set { , 1 , . . . , t } under the action of the powers of 
w is contained in the set, then the element Q m G S(gl m ) <g) ~P(U © U*) is divisible in 
the first tensor factor by the element of S(gl m ) corresponding to (4.31), where s is the 
length of the cycle. Then our argument reduces to a similar one with t replaced by t — s . 
Suppose that the w-cycle of each element of {0,1, ... ,t} is not contained in this set. 

Now consider the index w(t + 1) . If this index belongs to the set {t + 1 , . . . , t + p} 
then Q ro is divisible by the element of S(rj( m ) ®P (U®U*) corresponding to (4.25) with 
i = j = 1 , and our argument reduces to a similar one with p replaced by p — 1 . Suppose 
that w(t + 1) G {0,1,. ..,£}. Due to the symmetrization described above, without loss 
of generality we may assume that w(t + 1) = 0. Let s ^ be the minimal number such 
that w s+1 (0) <$l {0,1, ...,£}. Again due to (anti) symmetrization, we may assume that 

w(0) = 1, zu(l) = 2, ... , w(s — 1) = s and w(s) = t + l. 

Now we see that Q m is divisible by the element of S(gl m ) <S> P(U © U*) corresponding 
to (4.26) with i = j = l,a = CQ and b = Ct+i . This observation reduces our argument 
to a similar one where t + 1 and p are replaced by t — s and p — 1 respectively. □ 

Now let (G , G') be any of the reductive dual pairs from Subsection 4.1 with G = Sp 2m 
or G = 02m • Again, we will treat the cases of 9 = 1 and 9 = — 1 simultaneously. Denote 
by F the 2m x 2m matrix whose a , b entry is the matrix unit F ab G g . Here the indices 
a and b run through — m, . . . , — 1 , 1 , . . . , m. Regard the inverse matrix (x — 9 F) _1 as 
a formal power series in x~ x whose coefficients are 2m x 2m matrices with entries from 
the algebra U($j). Let [x — 9F)~ h be the a, b entry of the inverse matrix. It equals 

co m 

5 ab x- 1 +9F ab x- 2 + J2 Yl S+1 F aCl F ClC2 . . . F Cs _ lCs F Csb x~ s - 2 . 

8=1 |ci|,...,|c s | = l 

Like the indices a and b, here the indices c\ , . . . , c s run through — m, . . . , — 1,1, ...,m. 
In another guise, the next result appeared first in [02]. For connections to our present 
setting see [KN3, Section 6] or [KN4, Section 6] when 9 = 1 or 9 = — 1 respectively. 
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To state this result like Proposition 4.1, for a < put p a i = x _ 0j , and q a i = <9_ a ,i- 
For a > put = — 99id a % and (? a j = 9iX a i- Note that then our definition of the 
homomorphism £ : U(g) — >■ H(C/) for g = sp 2m 5 S0 2m can be written as 

n 

C(Fab) = 9S ab n/2 - ^2 OQaiPbi for a,b = -m, . . . , -1,1, . . . ,m. 

i=i 

Let U(g) G be the subalgebra of G-invariant elements in U(g) . This subalgebra coincides 
with the centre of U(g) if g = sp 2m , but is strictly contained in the centre if g = so 2m . 

Proposition 4.2. (i) One can define a homomorphism X(g') — > U(g) <g)H(E/) by 

m 

S i:i (x) ^ S i:j + (x±k + 0rn-6F)-l®p ai q bj . (4.33) 

|a|,|6|=l 

(ii) The subalgebra of all G-invariant elements in A = U(g) <S> H(C/) coincides with the 
subalgebra, generated by U(g) G ® 1 and 6y t/ie image of the homomorphism (4.33). 

Proof. Part (i) of the proposition was proved in [KN3, Section 6] and [KN4, Section 6] 
for 9 — 1 and 9 — — 1 respectively. The claim that the image of the homomorphism (4.33) 
belongs to the subalgebra A G C A, was also proved therein. It remains to prove that 
the elements 

m 

1 ®PaiQaj (4.34) 

|a| = l 

and 

m 

Y F aCl F ClC2 . . . F Cs _ lCs E Csb ^PaiQbj , (4.35) 

|o|,|6|,|ci|,...,|c a |=l 

taken together with the elements of U(g) G <8> 1 , generate the whole subalgebra A G C A. 
Here s = , 1 , 2 , . . . and i , j = 1 , . . . , n . For s = the first tensor factor of the summand 
in (4.26) should be understood as F ab . 

Consider the standard N-filtration on the algebra U(g) , where any element of g 
has degree 1. The adjoint action of the group G on U(g) preserves this filtration, and 
the corresponding graded algebra is identified with the symmetric algebra S(g). The 
algebra H({7) is generated by its subspace W = U © U*, which is identified with the 
tensor product C 2m ® C n . By presenting W as a direct sum of n copies of C 2m we get 
an N Xn filtration on the algebra H(U), such that the ith direct summand C 2m of W 
has degree 1 in the ith factor N of N Xn . This filtration is preserved by the action of 
the group G on H(U) . The corresponding graded algebra is identified with P(C 2m ) ® n . 
Here P(C 2m ) denotes the symmetric algebra of C 2m if 9 = 1, or the exterior algebra of 
C 2m if 6> = _i . it suffices to prove that the elements of the algebra S(g) ® P(C 2m )® n 
corresponding to (4.34) and (4.35), taken together with the elements of the subalgebra 

S(g) G ® 1 C S(g) ® P(C 2m ) ® n , (4.36) 

generate the whole subalgebra of G-invariants. We will prove this by using the classical 
invariant theory for the group G ; cf. [MO, Subsection 4.9]. 
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Now e a with a = —m,...,—l,l,...,m denotes a basis vector of C 2m . Consider the 
bilinear form on C 2m preserved by the action of the group G. Denote by e' a the vector 

signa-e_ a or e_ a 

when G is Sp 2m or 02 m respectively. The value of the form on a pair (e a ,e b ) equals 
S a b fc> r any indices a and b. The odd tensor powers of C 2m do not contain any non-zero 
G-invariant vectors. The G-invariant vectors in (C 2m )® 2 * are all linear combinations of 

m 

e Cl ®e' Cl ® ...® e Cp <g> e' Ct (4.37) 

|ci|,...,|c t | = l 

and of the sums, obtained from (4.37) by any simultaneous permutation of the 2t tensor 
factors of every summand. For instance, see [HI, Theorem IB]. 
First let us consider the G -invariants in the subspace 

l®P(C 2m )® n C S(g) ®P(C 2m )® n . (4.38) 

Regard the subspace of P(C 2m ) ® n of degree (si , . . . , s n ) G N Xn as a subspace in the 
tensor product 

(C 2m ) 8si <g> ... <g> (C 2m )® s ™. (4.39) 

Our subspace is obtained by applying to each of the n groups of the tensor factors C 2m 
of (4.39) the symmetrization or antisymmetrization, for 9 = 1 or 9 = — 1 respectively. 
By using the explicit description of G-invariants in (4.39) and arguing like in the proof 
of Proposition 4.1, one shows that the G-invariants in the subspace (4.38) are generated 
by elements of this subspace corresponding to (4.34) with i,j = 1 , . . . , n. For example, 
if n = 2t then (4.37) may be regarded as a element of the graded algebra P(C 2m ) ® n of 
degree (si , . . . , s n ) = ( 1 , . . . , 1) . Then (4.37) corresponds to the sum 

m 

^2 Pdi^di ■■■PcttQctt e H(C/). 

|ci|,...,|c t |=l 

Now consider the subspace (4.36). Generators of the subalgebra S($j) G C S(g) are 
well known. We will reproduce a set of generators here, because they will be used in the 
subsequent argument. If G = Sp 2m then q can be identified with the symmetric square 
of the G-module C 2m , so that F a (, e Q is identified with the element 

(e a ® + e' b ® e )/2 

of the symmetric square. If G = 02 m then q can be identified with the exterior square 
of the G-module C 2m , so that F ab e is identified with the element 

(e a ®e' b -e' b ®e a )/2 

of the exterior square. Now for G = Sp 2m , 02 m the subspace in S(g) of degree t becomes 
a subspace in the space of tensors (C 2m )® 2 *, and we may use the explicit description 
of G-invariants in the latter space. This description implies that the elements of the 
graded algebra S(g) corresponding to the elements 
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m 

^CiC 2 ^C 2 C 3 • • • ^C a _lC 3 ^C s Cl £ U(g) 

ci|,...,|c s | = l 

with s = 1 , 2 , . . . generate the subalgebra S(g) G C S(g) . A similar argument was given 
in the proof of Proposition 4.1 ; here we omit the details. 

Now take the subspace in S(g) ® P(C 2m ) ® n of degree t ^ 1 in the first tensor factor, 
and of any non-zero degree ( s\ , . . . , s n ) in the remaining n tensor factors. Regard it as 
a subspace in 

(C 2m )® 2t <g>(C 2m )® Sl <g> ... <g>(C 2m )® a ", (4.40) 

by using the embedding of g into the tensor square of C 2m . Our subspace is obtained by 
applying to each of the first t pairs of tensor factors C 2m of (4.40) the symmetrization 
or antisymmetrization for g = sp 2m or g = S02 m respectively, by symmetrizing these t 
pairs of tensor factors C 2m , and by applying to each of the n groups of the remaining 
tensor factors C 2m of (4.40) the symmetrization or antisymmetrization for 9 = 1 or 
$ = — 1 respectively. Using the explicit description of G-invariants in (4.40) and arguing 
like we did in the case G = GL m of Proposition 4.1, we complete the proof. □ 

4-4- Tensor products 

Recall the following general definition. Let F be any algebraic group over C with a Lie 
algebra f . Let E be any associative algebra where the group F acts by automorphisms. 
Consider the crossed product algebra F x E . It is generated by the elements of E and F 
subject to the relations in E and F together with the cross relations 

vZv' 1 = v(Z) for ZeE and v e F. 

Suppose there is also a homomorphism U(f) — > E. A module K over the algebra E and 
over the group F will be called a (E,F) -module, if the joint actions of E and F on K 
make it a module over the algebra F k E, and if the action of f on K corresponding to 
that of F coincides with the action of f obtained by pulling the action of E on K back 
through the homomorphism U(f) — > E. We suppose that F acts on K locally finitely. 

Take any reductive dual pair (G , G') from Subsection 4.1. We have U = C m Cg>C n . The 
Lie algebra g acts on the vector space P(f7) via the homomorphism ( : U(g) — > H(£/") . 
The action of the algebra H(£/") on P (U) is determined by (2.15). The group G' acts on 
H(i7) via linear transformations of the vector space C n . This action of G' preserves the 
subspaces U and U* of H({7) . It also leaves invariant any element in the image of the 
homomorphism £. Hence the actions of g and G' on P(U) commute with each other. 

Now for any pair of weights A , fx G t* consider the weight subspace 

(M M <g) P (U)) £ C (M M <g) P (U)) „ (4.41) 

of the space of n-coinvariants of tensor product of g-modules M M ® ~P(U) . We extend 
the action of the group G' from P(U) to the latter tensor product, so that G' acts on 
the tensor factor trivially. Then G' also acts on the pair of vector spaces (4.41). 
A bijective linear map 

P{U)^{M^®P{U)) n (4.42) 
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can be defined by mapping any «6P (U) to the coset of 1^ <8> u in (M^ ® P (U)) n . The 
algebra A acts on the vector space <g> P(U) by definition. By restricting the latter 
action to the subalgebra U(g) C A we get the diagonal action of g on M M ® ~P(U) , as 
above. When the A-module <g>P(U) is identified with A^ as in Subsection 2.3, then 
the pair of vector spaces (4.41) gets identified with the pair C M M . 

The group G' acts by automorphisms of the algebra A = U(g) ® H(C7) via its action 
on H (U) . The action of G' on A preserves each of the ideals I, J and J^' . Hence G' also 
acts on the pair C . Its identification with (4.41) is that of pairs of G'-modules. 

Note that the action of the Lie group G' on the subspace P(U) C H(C/) provides a 
homomorphism of algebras U(g') — > H(U). Any element of this image is G-invariant. 
For G' = GL n the image of the element E^ G gi n under this homomorphism equals 

(4.3) . For G' = O n or G' = Sp n the image of the element Eij — 9i9jE^ G g' equals 

(4.4) . By identifying H(U) with the subalgebra 1 ® H(C7) C A we get a homomorphism 
U(g') ->■ A G . Then (4.41) is a pair of (A G , G') -modules, and so is the pair C M M . 

More generally, for any element u of the group £H = & x X consider the subspace 

(M WOM ® P (£D (*7))) ^ oA C (M UOfl ®P(u)(U))) n (4.43) 

of the space of n-coinvariants of the tensor product of g -modules M WOM <S>P(u)(U)) . A 
bijective linear map 

P(u(U))^(M uoli ®P(u(U))) n (4.44) 

can be defined by mapping any w G P(Q(U)) to the coset of l WOA t ® w in the space of 
coinvariants. Similarly to (2.15) the action of the algebra H (U) on P(Q(U)) is defined by 

X{u) = v if Xu-v e R(U) -Q(U*) 

where u,v G P(Q(U)) . Then g acts on the vector space P (cD (U)) via the homomorphism 
C : U(g) — > H(CT') . The pair of vector spaces (4.43) gets identified with the pair 

MSjl 1 ap.)CM we)tit , (t ,.). (4.45) 

The action of G' on H(C/) commutes with that of G. In particular, the action of 
G' preserves the subspaces Q(U) and Q(U*) of H(U). Hence the group G' also acts 
on the two pairs (4.43) and (4.45). Our identification of them is that of pairs of G'- 
modules. Note that both (4.43) and (4.45) are also (A G , G')-modules, like they were in 
the particular case above, when to was the identity element of VI. 

The subalgebra A G of G-invariant elements of A acts on the weight subspace (4.43) 
by definition. First suppose that G = GL m . Like in the previous subsection, we will treat 
the cases of 9 = 1 and 9 = — 1 simultaneously. Via the homomorphism Y(gl n ) — > A m 
defined by (4.24), the subspace in (4.43) becomes a module over the Yangian Y(gl n ). 
We shall now describe this module explicitly, by using the comultiplication (4.10) and 
the homomorphism (4.8). For g = gi m the group % is trivial, while any cD preserves U 
and U*. Hence it suffices to consider only the case when u is the identity element. 

Every weight \i of g\ m is determined by the sequence \i\ , . . . , \i m of its labels where 
H a = n(E aa ) for each a = 1 , . . . , m. Note that here for the half-sum p of the positive 
roots p a = m/2 — a + \ . For g = gl m we get k — by the definitions (2. 12), (4.1). Then 
v = A — p by (2.18). Take the sequence v\ , . . . , v m of labels of v . If 9 = 1 then suppose 
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that each label v a G {0,1,2,...}. If 9 = — 1 then suppose that each v a G {0 , 1 , . . . , n } . 
Otherwise the weight subspace in (4.41) would contain zero only. 

If 9 = 1 then denote by <I? k the k th symmetric power of the defining 0[ n -module C n . 
If 9 = —1 then denote by the same symbol <P k the kth exterior power of the 0l n -module 
C n . The group G' = GL n also acts on <& k . Using the homomorphism (4.8), regard $> k as 
a module over the Yangian Y(gIJ . For teC denote by <P k the Y(gl n ) -module obtained 
by pulling the Y(gl n )-module <P k back through the automorphism (4.6) where z = 9t. 

For 9 = 1 denote by P (C n ) the symmetric algebra of the vector space C n . For 6 = — 1 
denote by P (C n ) the exterior algebra of C n . For any 9 and t the underlying vector space 
of the Y(rj[ n ) -module <3? k consists of all homogeneous elements of P(C n ) of degree k. 
Using the standard basis e\ , . . . , e m of C m , decompose the vector space U = C m ® C n 
into a direct sum of m copies of C n . Then the vector space P (U) gets identified with 
the tensor product of m copies of P (C n ) . For the proof of the following proposition see 
[KN1, Section 2] or [KN2, Section 2] when 9 = 1 or 9 = -1 respectively. 

Proposition 4.3. Under the above assumptions on the weight v , the subspace in (4.41) 
is equivalent as a Y(g[ n ) -module to the tensor product 

$"\ , i <8> ... <8> $" m , , i • (4.46) 

An equivalence map from the latter Y(Ql n )-module to the former is defined by regarding 
(4.46) as a subspace of P(U), and then applying the map (4.42) to that subspace. 

The equivalence map here is also that of GL n -modules. This follows from the GL n - 
equivariance of the map (4.42). The group GL n also acts by automorphisms of the Hopf 
algebra Y(gl n ) . When an element v G GL n is regarded as a n x n matrix by using the 
standard basis /i , . . . , f n of C n , the corresponding automorphism of Y(gl n ) is defined by 

T(x) H> v~ 1 T(x)v. 

The Olshanski homomorphism Y(gl n ) — > A GLm is GL n -equivariant. This can be verified 
directly by using the definition (4.24). Furthermore, we can consider the crossed product 
GL n k Y(gl n ) . Moreover, we have an embedding U(gl n ) — > Y(gl n ) . Then both modules 
in Proposition 4.3 become (Y(rj[ n ) , GL n ) -modules. For the latter module this can be 
verified directly, by using the definition of <& k . For the former this follows from GL n - 
equivariance of (4.24), by regarding £g> P (U) as a module over the algebra GL n k A. 

Now take u = a with arbitrary a G & . By replacing in Proposition 4.3 the weights A 
and fx respectively by ao A and tro^, we get a description of the subspace in (4.43) with 
u = a as a module over the Yangian Y(gl n ) . Then v is replaced by o{v) . Let us identify 
the Weyl group & with the symmetric group & m so that any permutation of the dual 
basis vectors r/i , . . . , n m G t* by an element a G & corresponds to a permutation of the 
numbers 1 , . . . , m . The latter permutation will be denoted by the same symbol a . Put 

fta = Ha- 1 ^), V a = V a -i( a ), Pa = Pa- 1 (a)- 

These are the ath labels of the weights a(fx) , a(u) , a(p) respectively. In this notation, 
the subspace in (4.43) with u = a is equivalent as a module over Y(g[ ri ) and GL n to 
the tensor product 
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, i <g> ... <g> j.i • ( 4 - 47 ) 

Pl+Pl+2 

Now suppose that the weight A + p of rj( m is nonsingular. By Corollary 3.6, then the 
Zhelobenko operator £ a on A/ J defines a linear map (3.15). This map commutes with 
the actions of the algebra A GLm on the source and the target vector spaces of (3.15). 
Hence it is an intertwining operator of Y(gl n ) -modules. By replacing the modules by 
their equivalents, we now get an intertwining operator 

$ v \ j.!®...®*"™ ,i ->• <8> ... ® ,! (4.48) 

A*l+Pl+2" + 5 Ml+Pl + 5 A*m+Pm + 5 V 7 

of two tensor products of Y(g[ n ) -modules. It is well known that both tensor products 
are irreducible and equivalent to each other if (but not only if) the weight p is generic, 
that is if n a — p ^ Z whenever a ^ b; see [NT2, Theorem 4.8] for a more general result. 
Hence for generic p, an intertwining operator between the two tensor products is unique 
up to scalar factor. For our particular intertwining operator, this factor is determined by 
the next proposition. This proposition is valid for any weight p , not necessarily generic. 

Choose ifk G @ k as follows. If 9 = 1 then @ k is the feth symmetric power of C n , 
and we put <pfc = If = —1 then is the feth exterior power of C n , and we put 
<Pfc = fi A . . . A fk . The vector is annihilated by the action of the elements Eij e 0l n 
with i < j . Take any positive root a = n a — rjbof Ql m with a < b. For 6* = 1 define 

nPa - Pb + Pa - Pb - S 

Here the denominator corresponding to the running index s equals (A + p){H a ) + s. 
Hence the denominator does not vanish for any nonsingular A + p. For 9 = — 1 define 

A a — A + p a — 

■ if v a <v h ; 

(-l)"-"* <j Pa- Hb + Pa- Pb 

1 if Z/ a ^ Z/ b . 

Here p a = X a — v a for any index a. Therefore in the first of the last two cases, the 
denominator equals (A + p) (H a ) — u a + Ub- Hence it does not vanish for any nonsingular 
weight A + p, under the condition v a < v\> when this denominator occurs. 

Proposition 4.4. Let X + p be nonsingular. The operator (4.48) determined by £ CT maps 
the vector <p Ul <S> ■ ■ ■ <S> <f Vm of (4.46) to the vector <p ^ <g> . . . <S> <fi t> m of (4.47) multiplied 
by all those z a where a G A + but a (a) A + . 

Proof. For a generic weight p, this proposition has been proved in [KN1, Section 3] and 
[KN2, Section 3] when 9 = 1 or 9 = — 1 respectively. But when the weight v is fixed, 
our operator (4.48) depends on p e t* continiously, see the proof of Corollary 3.6. □ 

For any p and nonsingular A+p, Corollary 3.9 and Proposition 4.1 give that for a = o~o 
the quotient by the kernel, or equivalently the image of our intertwining operator 

$ V1 . , i ® ... (S^"™ xl -> $ Vm , ,! <g> ... <g> ® v \ ,! (4.49) 

A*l+Pl+2 Vm+Pm+2 Pm+P m +2 Pl+Pl+2 
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is an irreducible Y(gi n ) -module. Here we also use the observation that any element of the 
subalgebra U(gl m ) GLm ® 1 C A GLm acts on the subspace (4.41) by scalar multiplication. 
In the next section we will show that up to equivalence and similarity, every irreducible 
finite-dimensional Y(gl n ) -module arises as the image of (4.49) for 9 = — 1 and some A , \x . 

Now let (G , G') be any of the reductive dual pairs from Subsection 4.1 with G = Sp 2m 
or G = 02 m • We will continue treating the cases of 9 = 1 and 9 = — 1 simultaneously. 
Via the homomorphism X(g') — » A G defined by (4.33), the subspace in (4.43) becomes 
a module over the extended twisted Yangian X(g') . We shall now describe this module 
explicitly. For g ^ gl m we do not have Q (U) = U in general. We will give an analogue of 
Proposition 4.3 for arbitrary u G 9t. For the proof of this analogue see [KN3, Section 5] 
and [KN4, Section 5] in the cases 9 = 1 and 9 = — 1 respectively. We will keep using 
the Y(gl n ) -modules <3?t for integers k ^ 0. Further, denote by <&t k the Y(gl n ) -module 
obtained by pulling the Y(gl n ) -module @ k back through the automorphism (4.12). 

We regard g' is a Lie subalgebra of gl n . Any Y(gl n ) -module can also be regarded as 
a X(g') -module, first by restricting from Y(gl n ) to the subalgebra Y(g'), and then by 
pulling back through the homomorphism X(g') — > Y(g') defined by (4.17). The resulting 
X(g') -module can also be defined as the tensor product of the initial Y(gl n ) -module 
by the one-dimensional trivial X(g') -module, defined by the assignment Sij(x) >->■ Sij. 
Here we use the coaction of Y(gi n ) on X(g') defined by (4.23). Either way, any tensor 
product of Y(gl n ) -modules of the form $ k or <£^~ will be regarded as a X(g')-module. 
The group G' acts on and on <&~[ k by restricting the natural action of GL n on <& k . 

For any weight fx of g = sp 2m or g = so 2m define the sequence \i\ , . . . , (i m of its labels 
by setting \i a = /j (F aa ) for a = 1 , . . . , m. Note that unlike in the case of g = gl m , here 

= -HmVi ~ ■ ■ ■ ~ ViVm- (4.50) 

For the half-sum of positive roots we get p a = — a if g = sp 2m , or p a = 1 — a if g = &0i m ■ 
By the definitions (2. 12), (4. 2) here K a = 9n/2. Consider the sequence v\ , . . . , v m of the 
labels the weight (2.18). Like for g = gi m , suppose that each label v a e {0,1,2, .. .} 
if 6 = 1. If 6 = — 1 then suppose that each v a G {0,1,..., n} . Otherwise the weight 
subspace in (4.43) would contain zero only. 

The Weyl group of sp 2m is isomorphic to the semidirect product 6 m ix , and the 
group X is trivial in this case. The Weyl group of S02 m is isomorphic to a subgroup of 
& m k of index two, and % is isomorphic to Z 2 . The extended Weyl group 91 = & k X 
of 502m is isomorphic to & m ix Z™. Fix the isomorphisms as follows. Regard & m ix Z^ 
as the group of permutations of the indices — m, . . . ,— 1,1, ... ,m such that if a (->■ b 
under a permutation, then — a h- > — b under the same permutation. The image of ui G 91 
in & m k Z^ will be denoted by u . Then for any c = 1 , . . . , m — 1 the permutation a c 
only exchanges c — m — 1 with c — m , and m — c with m — c + 1 . For g = sp 2m the 
permutation a m exchanges only —1 and 1. For g = S02m the transposition of —1 and 1 
will be denoted by f m ; then a m = f m a m -i f m . The transposition f m is the image in 
& m k Z^ of the generator r m of X for g = so 2m . Now put 

fra = A»|o-i(a)| > i 'a = V\a- 1 (a,)\> Pa = P\Cj-^(a)\ and S a = sign U _1 (a) 

for a = 1 , . . . , m . In this notation, the a th label of the weight u (p) is equal to 8 a £t a • 

For any a = 1 , . . . , m let u a be the element of the group G such that cD a (e_ a ) = e a , 
and u a (e a ) = -e_ a or w a (e a ) = e_ a depending on whether G = Sp 2m or G = 2m . By 
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definition, all other basis vectors of C 2m are invariant under the action of u a . We have 
u a G NormT. The image of Q a in the group £R will be denoted by u> a . In particular, 
wi = a m in the case G = Sp 2m , but u\ = r m in the case G = 02 m - The action of the 
element Q a G G on the vector space W = C 2m <g> C n determines an automorphism of 
the algebra H (U) such that for alH = 1 , . . . , n 

x ai 1 ^ @ @i d ai and d a i i y 9i x a \ , 

while the elements xu , du G H (U) with b ^ a are invariant under this automorphism. 
This automorphism of H({7) will denoted by the same symbol oj a . 

Proposition 4.5. Under the above assumptions on the weight v , the subspace in (4.43) 
is equivalent as module over X(g') to the tensor product 

5lt 2~ j. i ® • • • ® * i (4.51) 

pulled back through the automorphism (4.15) o/ X(g') where 



m n 1 

/(x) = TT ^-p--2 

a=! 6% ~ Pa+ 2 



An equivalence map from the latter X(g') -module to the former is defined by regarding 
(4.51) as a subspace of P(U) C H({7), t/ien applying to this subspace all automorphisms 
a;" 1 wit/i 5 a = — 1, arac? ^/ien applying the map (4.44) to £/ie resulting subspace of H(U). 

The equivalence map here is also that of G'-modules. This fact follows from the G'- 
equivariance of the map (4.44). The group G' also acts by automorphisms of the right 
coideal subalgebra Y(g') C Y(gl n ) , and by automorphisms of the right Y(gl n ) -comodule 
algebra X(g'). When an element v G G' is regarded as a n x n matrix by using the 
standard basis /i , . . . , f n of C n , the corresponding automorphism of X(g') is defined by 

S(x) ^ v- 1 S(x)v. (4.52) 

It factors to an automorphism of the quotient Y(g') of the algebra X(g') . The Olshanski 
homomorphism X(g') — y A G is G'-equi variant. This can be verified directly by using the 
definition (4.33). We can consider the crossed product algebra G' x Y(g') . We also have 
an embedding U(g') — > Y(g') , defined by mapping each element Eij — OiOjEji G g' to 
the coefficient at x~ x of the series (4.13). Then (4.51) becomes a (Y(g') , G') -module. 
This statement can be verified directly, by using the definitions of <f>\ and <3>^ k . 

Note that the series f(x) in Proposition 4.5 does not depend on the choice of u G 
By using this proposition in the basic case when u is the identity element, the subspace 
in (4.41) is equivalent as an X(g') -module to the tensor product of the form (4.46) pulled 
back through the automorphism (4.15) of X(g') where the series f(x) is as above. But 
here the labels (i a , v a , p a for a = 1 , . . . , m correspond to the Lie algebra g = sp 2m or 
g = S02m, not to g = gl m . In particular, here the labels p a of the half-sum p of the 
positive roots are different from those for g = g[ m . Our choice (4.50) of the labels of \x for 
g = 5p 2m or g = S02 m has been made so that here the description of the subspace (4.41) 
becomes similar to that in the case g = gl m . For the same purpose, in the definition 
(4.24) we employed the matrix (x + 0m/2 + 6 E) _1 rather than the matrix (x + 6E) _1 . 
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Now we suppose that the weight A + p of g = sp 2m or g = S02 m is nonsingular. Then 
by Corollary 3.6, the Zhelobenko operator £ CT on A/ J determines a linear map (3.15). 
This map commutes with the actions of the algebra A G on the source and target vector 
spaces of (3.15). Further, for g = S02 m and u = r m a the representative f m G G of r G I 
yields a linear map 

M£jU(tf.) M^^, (4.53) 

see (3.2). This map also commutes with the actions of A G on the source and the target 
vector spaces. Note that r m G 02m is an involution, and the corresponding map (4.53) is 
invertible. Take the composition of the latter map with the map (3.15) determined by £ CT . 
Thus for g = sp 2 m or = 30 2m and for any u G Ut we get a linear map 

K M £o°i,s ( t/*), (4-54) 

commuting with the actions of the algebra A G on the source and target vector spaces. 
Hence this is an intertwining operator of X(g') -modules. By replacing the modules in 
(4.54) by their equivalents, and by using the observation that the series f(x) for both 
modules is the same, we get an intertwining operator of X(g') -modules 

$ v \ ,®...®<^"\ , ->■ $ 5li 2~ ® ■■■ ® $t mt> ™ ^ . (4.55) 

/^l+Pl+2 Hm+Pm+2 Ml+Pl+2 Mm+Pm+g 

The last two can also be regarded as Y(g') -modules, and our operator intertwines them. 
In this case we first take the tensor products of Y(gl n ) -modules, and then restrict both 
tensor products to the subalgebra Y(g') C Y(gl n ) . 

We will now give an analogue of Proposition 4.4 for G = Sp 2m , 02 m • Let us arrange 
the indices 1 , . . . , n into the sequence 

1,3,. ...,4,2 or 1 ,3, . . . , n - 2,n,n - 1 , . . . , 4,2 (4.56) 

when n is even or odd respectively. The mapping i i— >■ i reverses the sequence (4.56). We 
will write i -< j when i precedes j in this sequence. The elements Eij — 9i9jEji G Qi n 
with i -< j or i = j span a Borel subalgebra of g' C gt n , while the elements En — En 
span the corresponding Cartan subalgebra of g' . Choose a vector ipk G $ k as follows. 
For 9 = 1 put ipk = ft 7 so that (pk = ipk in this case. However, for 9 = — 1 let ipk be the 
exterior product of the vectors fi taken over the first k indices in the sequence (4.56). 
For instance, 1(^2 = fi A if n ^ 3 . Note that the vector ifjk is always annihilated by 
the action of the elements E^ G gl n with i < j . 

For each positive root a of g = 5p 2 m or = S0 2m define z a G C as follows. If 9 = 1 , 



Pa- Vb + Pa~ Pb~ S 



n 



s 



S 



if a = i] m - b+1 - 7] 



m—a+l 5 



I TT Pa + Pb + Pa + Pb + S 

z a = \ t — : ; ; n a = i] m - b+1 + i] m - a+1 , 

^ A a + X b + p a + p b - S 

K/2] 

nPa + Pa + S 
; if a = 2 7] m - a +l ■ 

K s -i ^a + Pa-S 
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In the first two cases we have 1 ^ a < b ^ m, while in the third case 1 ^ a ^ m and 
= sp2m • I n the first case, the denominator corresponding to s equals (A + p)(H a ) + s , 
while in each of the last two cases the denominator equals — (A + p){H a ) — s. All these 
denominators do not vanish for a nonsingular A + p. If 9 = —1, then z D 



z 'a z a where 




if a = 7] m - b+1 
otherwise ; 



( A a - X b + Pa - Pb 
P>a - P>b + Pa - Pb 
Aq + Xb + Pa + Pb 
Pa + Pb + Pa+Pb 
Aq + Pa 
Pa+ Pa 
1 



if a 
if a 



Vm-a+i or a = r) m - b+1 + i] 

: rjm-b+i - Vm-a+i and v a < v h , 
= Vm-b+i + Vm-a+i and v a + v h > n , 

and 2 v a > n , 



if a = 2?] m _q + i 
otherwise . 



Here for 9 = —1 we have /i a = A a — v a + n/2 for each index a. Hence in the first of the 
last four cases, the denominator equals (A + p){H a ) — v a + v h . In the second of these 
four cases, the denominator equals — (A + p)(H a ) — v a — Vb + n. In the third case, the 
denominator equals — (A + p)(H a ) — v a + n/2. These denominators do not vanish for 
any nonsingular A + p, under the conditions they occur with. Denote by A ++ the set 
of compact positive roots of q , these are the weights r\ a — f]b where 1 ^ a < b ^ m . 

Proposition 4.6. Let X + p be nonsingular. The operator (4.55) determined by £ CT maps 
the vector 

(4.57) 



Mm ~~\~pm. H~ 2 



to the vector ip^ <S> ■ ■ ■ <S> ipi> m of (4.51) multiplied by those z a where u>(a) A + while 

f A++ if 9 = 1 and n > 1 , 



a G 



otherwise. 



Proof. For a generic weight fx, this proposition has been proved in [KN3, Section 5] and 
[KN4, Section 5] when 9 = 1 or 9 = — 1 respectively. But when the weight v is fixed, 
our operator (4.55) depends on p e t* continiously, see the proof of Corollary 3.6. □ 

Our intertwining operator of Y(rj') -modules (4.55) has been defined for any element 
u G D\. By definition, the corresponding element u; G 6 m xZ™ is a certain permuation of 
the indices — m, . . . , — 1 , 1 , . . . , m. Now consider the special case when the permutation 
cD only changes the sign of each of these indices, so that u = u>i . . . u m in the notation 
introduced just before stating Proposition 4.5. Thus we get an intertwining operator 



A*i+Pl + 5 



7) 



(4.58) 



of Y(fl') -modules. It corresponds to the longest element ui = o"o of the Weyl group & , if 
= 8p 2m or if g = 502m and m is even. If g = S02m and m is odd, then o~o = ui ■ ■ ■ 0J m 
so that (4.58) corresponds to the element u = r m a of the extended Weyl group £H. 
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For any g = sp 2m ,502m the element 0J2 ■ ■ ■ uj m G 9\ gives the intertwining operator 
$ v \ , 1 ® ...gxP"" 1 , , t ->• <f> v \ ^®$~ v l ^®...®<f>- v ™ xl (4.59) 

A*l+Pl+2 V-m+Pm+2 (J-l+Pl+2 P2+P2+2 Vm+Pm+2 



of Y(g')-modules. The underlying vector spaces of the two Y(g[ n ) -modules appearing 
as the first tensor factors on the right hand sides of (4.58) and (4.59) are the same by 
definition. Moreover, for g = so 2m our operators (4.58) and (4.59) are the same due to 
Proposition 4.5. In particular, for g = SO2 the intertwining operator 

Ml+Pl+2 P-l+Pl+2 

corresponding to u = u\ is the identity map. This explains the following fact from [N]. 
If g' = sp n and 6 = 1 , or if g' = so n and 6 = — 1 , that is if g = 502m in the context of 
the present article, then for any i G C the restriction of the Y(gl n ) -module (£>^~ k to the 
subalgebra Y(g') C Y(gl n ) coincides with the restriction of the Y(gl n ) -module <&t . 

Now suppose that g = sp 2m ■ Then we have £K = & , so that 9l\ = &\ for any A G t* 
automatically. Thus for any p and nonsingular A + p, Corollary 3.9 and Proposition 4.2 
imply that the quotient by the kernel of our intertwining operator (4.58) is an irreducible 
Y(g') -module. Here we also use the observation that any element of the subalgebra 
U(-Sp2m) Sp2m ® 1 C A Sp2m acts on the subspace (4.41) via scalar multiplication. Note 
that here g' = so n or g' = sp n respectively for 9 = 1 or 9 = — 1 . In the next section 
we will show that up to equivalence and similarity, any finite-dimensional irreducible 
Y(sp n ) -module arises as the image of (4.58) for 6 = — 1 and some A, \x. 

Next suppose that g = S02m- Then 7^ &. Here for any \i and nonsingular A + p, 
Corollary 3.9 and Proposition 4.2 imply that the quotient by the kernel, or equivalently 
the image of our intertwining operator (4.58) is an irreducible Y(g') -module, under the 
extra condition that D\\ = &\ . We also use the fact that any element of the subalgebra 
U(£02 m )° 2m <8> 1 C A° 2m acts on the subspace (4.41) via scalar multiplication. In the 
next subsection we study the quotient by the kernel of (4.58) without imposing that 
extra condition, but for 9 = — 1 only. We will show that then the quotient is either an 
irreducible Y(so n ) -module, or splits into a direct sum of two non-equivalent irreducible 
Y(so n ) -modules. In Section 5 we will explain which irreducible Y(so n ) -modules arise 
in this particular way. 

4-5. Crossed product algebras 

For any pair (G, C) consider the crossed product algebra G'kA. The action of G' on A 
commutes with that of the group G , and leaves invariant any element of the subalgebra 
U(g) C A. Therefore the group G' acts by automorphisms of the double coset algebra 
Z, and we can also consider the crossed product algebra G' x Z. 

The action of G' on Z commutes with the Zhelobenko automorphisms |i , . . . , | r . 
Since it also commutes with the action of r G G on Z for any r G X , the action of G' 
preserves the subalgebra Q C Z. So we get the crossed product algebra G' k Q. The 
homomorphism U(g') — > A G used in Subsection 4.4 yields a homomorphism U(g') — > Q. 

For any \,p, G t* the subspace C M M is a module over the subalgebra 



G' x A G c G' x A, 
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and moreover an (A , G')-moduie. If A + p is nonsingular, the subspace C M M is 
also an Q -module, and moreover an (Q , G') -module. Then the action of A G on 
can also be obtained by pulling the action of Q on back through the isomorphism 
7 : A G — y Q, see Subsection 3.3. Moreover, the isomorphism 7 is G'-equivariant. Hence 
by using 7 we get the same structure of an (A G , G') -module on as above. 

The Zhelobenko operator £ on A/ J is G'-equivariant. So is the corresponding linear 
map (3.16) for any ji and nonsingular X + p. Hence the action of the group G' preserves 
the subspace Ker (£0 I ) of . Another way to see this is to use Proposition 3.7 and 
G'-contravariance of the Shapovalov form on . The latter property means that 

S^v(f),g) = S^f,v\g)) for f,geM* 

where v 1— >■ v' is the anti-involution on G' defined by the matrix transposition. Here the 
elements of G' are regarded as matrices by using the standard basis f\ , . . . , f n of C n . 
Either way, the quotient space of by Ker(£ |M*) becomes an (Q,G') -module. 

When the Lie group G' is connected, that is when G' = GL n or G' = Sp n , we will 
not need to use the action of G' x Q on . It will suffice to use only the action of Q . 
But the Lie group G' = O n is not connected. Until the end of this subsection we will 
be considering only the case when (G,G') = (0 2m ,O n ) so that 9 = — 1. The space U 
has been identified with the tensor product C m ® C n . In our case H(C7) is the Clifford 
algebra generated by the elements of the vector space W = C 2m ® C n subject to the 
relations (2.2), where B is the tensor product of the symmetric forms on C 2m and C n 
preserved by the actions of the groups 02 m and O n respectively. Here x a i = e a ® fi and 
d a i = e- a <S> ft for a = 1 , . . . , m and for alH = 1 , . . . , n . Note that 9i = 1 in this case. 

Choose any vector f e C n of length \/2 with respect to the form preserved by O n . 
The corresponding orthogonal reflection is 

w : C n -)■ C n : u ^ u-zfo 

where z is the value of the symmetric form on C n taken on the pair of vectors (fo,u) . 
This reflection is an element of the group O n . It determines an automorphism of the 
algebra H (£/"), which preserves the subalgebra ~P(U) generated by all the elements x ai . 

For a = 1 , . . . , m define the vectors x a = e a <8> f and d a = e_ a ® f of C 2m ® C n . 
By (2.2) we get 

x a d a + d a x a = B(x a ,d a ) =2. (4.60) 

Consider the product 

m 

A = ]J(l-x a d a ) e R(U). 

a=l 

The m factors of this product pairwise commute. We will use the next properties of Aq . 
Lemma 4.7. (i) There is an equality f m (Ao) = — Aq . 

(ii) The action of the element Aq on P(U) coincides with that of the reflection vq . 

(iii) The element A e H(C/) commutes with ((X) for every X e 502 m . 

Proof. In our case, the action of the element r m G 02m on C 2m exchanges e± with e_i , 
and leaves other basis vectors of C 2m fixed. So the action of r m on H(C7) exchanges the 
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element x\ with d\ , and leaves fixed any element x a or d a with a > 1. Now (i) follows 
from the relation (4.60) with a = 1 . Part (ii) can be obtained by a direct calculation, 
which reduces to the case m = 1. Part (iii) follows from (ii), because every element 
((X) G H(f7) with X G S02 m is O n -invariant. □ 

Proposition 4.8. Suppose that X + p is nonsingular and (G,G') = (02 m ,O n ) . Then 
for any weight \x G t* the quotient / Ker(£ I M^) is an irreducible (Q,O n ) -module. 

Proof. By Propositions 2.6 and 3.7 the quotient iV = / Ker(£o I ) is an irreducible 
S -module. By applying Proposition 1.6 to this S -module, we get Proposition 4.8 when 
= &\ . In this case Proposition 4.8 follows from Theorem 3.8. We will now modify our 
proof of Proposition 1.6, to get Proposition 4.8 without assuming that ${\ = &\. 
Instead of the element (1.23) of the algebra Q let us first consider an element of S, 

z' = \e x \- 1 Y J Ux'Y). 

Here X' and Y are the same as in (1.23), but the sum is taken over the Weyl group 6 . 
The arguments given just after (1.23) show that x(Y) = x(Z') • We also get the equality 
^cr(Z') = Z' for every a G & . But we might have r m (Z') ^ Z', so that Z' ^ Q then. 

Then put Z = Z' +r m (Z') . Since = 1 , we get the equality r m (Z) = Z . The action 
of the element r m G 02 m on S exchanges the operator £ m _i with £ m , and commutes 
with the operators £i , . . . , £, m -2 ■ Hence ia{Z) = Z for any a G & . Thus we have ZeQ, 
like we had for the element (1.23). 

Now put Z" = Z'-Z. Then £ a {Z") = Z" for any a G 6 , while ? m (Z") = -Z". But 
here we have Z"A G Q. Indeed, by Part (iii) of Lemma 4.7 we have £ a (Z"A ) — Z"A 
for any a G &, while r m {Z" Aq) = Z"Aq by Part (i). Now consider the element 

Z'^oVq 1 G O n x Q. 

Its action on iV coincides with that of the element Z" due to Part (ii) of Lemma 4.7. 
Hence the action of 

Z + Z"A G v^ G O n x Q 
coincides with that of the element Z' . But the latter action coincides with that of Y. □ 

Corollary 4.9. Suppose X+p is nonsingular and (G , G') = (02m, O n ) , so that 6 = — 1 . 

(i) If n is odd then the quotient / Ker(£ I ) is an irreducible A° 2m -module. 

(ii) If n is even then / Ker(£ I ) is either an irreducible A° 2m -module, or splits 
into a direct sum of two irreducible non-equivalent A° 2m -modules. 

Proof. If n is odd then the group O n splits as a direct product Z2 x SO n where Z2 is the 
subgroup of O n generated by the minus identity element. This element acts on as the 
multiplication by (— 1)^1 + •••+^. Instead of the action of the connected Lie group SO n 
on it suffices to consider the action of the Lie algebra so n . But the latter action can 
also be obtained by pulling the action of A 2m on back through the homomorphism 
U(50 n ) — > A° 2m , as mentioned earlier in this subsection. Proposition 1.5 now yields (i). 

When n is even, we can use the action of the Lie algebra so n on instead of the 
action of the subgroup SO n C O n . This is a normal subgroup of index two. We can also 
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use the homomorphism U(50 n ) — > A 2m , as we did for an odd n. Proposition 1.5 now 
implies that the quotient / Ker(£o I ) is irreducible over the joint action of A 2m 
and of any element v from the complement to SO n in O n . Now general arguments from 
[W, Section V.8] yield (ii). Moreover, if the quotient is a direct sum of two irreducible 
A° 2™ -modules and N is one of them, the space of the other equals vN for any v. □ 

4-6. Equivalent modules 

Let (G,G') be any dual pair from Subsection 4.1. For any weights A and p of 0, the 
subspace (2.31) can be regarded as an A G -module. We have denoted the subspace by . 
First consider (G,G ; ) = (GL m ,GL n ). By using the homomorphism Y(gl n ) — > A GLm 
we can also regard L^ as an Y(gl n ) -module. By Corollary 2.8 the Y(gl n ) -module L^ is 
irreducible for any p and nonsingular A + p. Here we also use Propositions 1.5 and 4.1. 

Further suppose that the weight v = A — p satisfies the conditions of Proposition 4.3. 
Using that proposition, the Y(gl n ) -module is equivalent to a certain quotient of 
the tensor product (4.46). The latter tensor product can be regarded as a subspace in 
P (U) . Denote by u the vector of P (U) corresponding to the vector (p Ul <g> . . . <g> (p Vm of 
(4.46). If the weight A + p is nonsingular, then C,{P\p + p])u = zu where z stands for 
the product of all z a with a G A + . This equality follows from Propositions 3.5 and 4.4. 
It can also be obtained directly from [KN1, Section 3] or [KN2, Section 3] when 9 = 1 
or 9 = — 1 respectively. If z 7^ then by Propositions 2.9 and 3.7 the Y(gl n ) -module L^ 
is equivalent to the image of our intertwining operator (4.49). Then L^ is not zero. 

Now take any pair (G, G') from Subsection 4.1 other than (GL m , GL n ) . By using the 
homomorphism X(rj') — > A G we can regard L^ as an X(fj') -module. Moreover, L^ can 
be then regarded as a (X({j'), G') -module, see the beginning of Subsection 4.5. Take 
any p and nonsingular A + p. By Corollary 2.8 the X(rj') -module is irreducible, if 
£Ha = ©a • Here we also use Propositions 1.5 and 4.2. Next take (G, G') = (02 m , n ) , so 
that 9 = — 1. Then our proof of Proposition 4.8 demonstrates that is an irreducible 
(X(so n ) , O n ) -module. Our proof of Corollary 4.9 demonstrates that is an irreducible 
X(so n ) -module, if n is odd. If n is even then is either an irreducible X(so n ) -module, 
or a direct sum of two irreducible non-equivalent X(so n ) -modules. 

Now consider again any pair (G,G ; ) from Subsection 4.1 other than (GL m ,GL n ). 
Suppose that the weight v = A — p — k satisfies the conditions of Proposition 4.5. Using 
that proposition in the case when u is the identity element of the group the X(g')- 
module is equivalent to a certain quotient of the tensor product of the form (4.46). 
This tensor product can be regarded as a subspace in P (U) . Let u be the vector of 
P(U) corresponding to (4.57). If A + p is nonsingular, then we again have the equality 
C(P[p+p])u = zu. But here z stands for the product of all z a with a G A ++ if 9 = 1 and 
n > 1 . Otherwise z stands for the product of all z a with a G A + . The equality follows 
from Propositions 3.5 and 4.6. It can also be obtained directly from [KN3, Erratum] or 
from [KN4, Section 5] when # = lor# = — 1 respectively. If z 7^ then by Propositions 
2.9 and 3.7 the X(rj') -module L^ is equivalent to the image of our intertwining operator 
(4.58). Moreover, then L^ is equivalent to the image of (4.58) as an (X(g') , G') -module. 
Note that then the quotient L^ of M^ is not zero, see Subsection 2.5. 
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4-7. Dual modules 

In Subsection 3.4 for any \i and nonsingular A + p we denned a Q-contravariant pairing 
(3.19). Let us now describe that pairing in terms of representations of Yangians. First 
take (G, G') = (GL m , GL n ) . The Chevalley anti-involution e on the Lie algebra g = Ql m 
is defined by e (E ab ) = E ba • It extends to an involutive anti-automorphism of the algebra 
A, also denoted by e, so that the extension exchanges the generators x ai and d ai of the 
subalgebra H(£Z) C A. First applying the homomorphism Y(gl n ) — y A defined by 
(4.24), and then applying the anti-automorphism £ of A, amounts to first applying 
the anti-automorphism (4.11) of Y(rjLj, and then applying the homomorphism (4.24). 
Therefore, when we regard the two vector spaces in the pairing (3.19) as Y(gl n ) -modules, 
the pairing becomes contravariant relative to the anti-automorphism (4.11) of Y(gl n ). 

By replacing these two Y(gl n ) -modules by their equivalents, we get a non-degenerate 
contravariant pairing of the source and target Y(gl n ) -modules in (4.49). In particular, 
the Y(gl n ) -module dual to the source in (4.49), is equivalent to the target. The latter 
equivalence can be proved directly, for all A and fx. Indeed, because (4.11) is a coalgebra 
anti-automorphism, it is enough to consider the case m = 1 only. But it is well known 
that the Y(gl n ) -modules <f>\ are equivalent to their duals relative to (4.11), whenever 
= lor0 = — 1. See [NT1, Proposition 1.7] for a more general result. 

Now let (G,G') be any pair from (0.1) other than (GL m ,GL n ). The Chevalley anti- 
involution e on the Lie algebra g is defined by setting for a , b = — m, ... ,—1,1, ... ,rn 

e (Fab) = sign ab ■ F ba or e (F ab ) = F ba 

when 9 = 1 or 9 = —1 respectively. It extends to an involutive anti-automorphism of the 
algebra A, also denoted by e, so that the extension exchanges the element p m G H(CT') 
with the element 

- sign a ■ q ai or q ai 

when 9 = 1 or 9 = — 1 respectively. Here we use the notation from Proposition 4.2. 

First applying the homomorphism X(g') — y A defined by (4.33), and then applying 
the anti-automorphism e of A , amounts to first applying the anti-automorphism (4.20) of 
X(g') , and then applying the homomorphism (4.33). Therefore, when we regard the two 
vector spaces in the pairing (3.19) as X(g') -modules, the pairing becomes contravariant 
relative to the involutive anti-automorphism (4.20) of X(g') . 

By replacing these two X(g') -modules by their equivalents, we get a non-degenerate 
contravariant pairing of the source and target X(g') -modules in (4.58), if g = sp 2m 
or if g = 502m and m is even. If g = S02 m and m is odd, then we get a pairing of 
the source and target X(g')-modules in (4.59). But if g = S02 m , then the target X(g')- 
modules in (4.58) and (4.59) are equivalent, see the end of Subsection 4.4. Thus for both 
g = 8p 2rn ,S02m and for any m we get a non-degenerate contravariant pairing of the 
source and target X(g')-modules in (4.58). Let us now regard them as Y(g') -modules, 
like we did in Subsection 4.4. Then we get a non-degenerate pairing of them, which is 
contravariant relative to the restriction of the involutive anti-automorphism (4.19) of 
Y(gl n ) to the subalgebra Y(g') C Y(gl n ) . We also use the fact that any automorphism 
of X(g') defined by (4.15) commutes with the anti-automorphism defined by (4.20). In 
particular, the Y(g') -module dual to the source in (4.58), is equivalent to the target. 
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Like in the case g' = gl n , the equivalence here can be proved for all A and \i directly. 
Indeed, the Y(rj') -module dual to the source in (4.58) can be defined by first considering 
the source as a Y(gl n ) -module, then taking its dual relative to the anti-automorphism 
(4.19) of Y(gl n ), and then restricting the resulting Y(gl n )-module to the subalgebra 
Y(g') C Y(g[J. But (4.19) is the composition of (4.11) and (4.12). Hence taking the 
dual relative to (4.19) amounts to first taking the dual relative to (4.11), and then 
pulling the result back through the automorphism (4.12). 

We have already proved that relative to (4.11), the Y(gt n )-module dual to the source 
tensor product in (4.58) is equivalent to the tensor product of the same factors, but taken 
in the reversed order. Pulling the latter tensor product back through (4.12) replaces each 
factor by <3?^ k , and also reverses the order of the factors once again. Thus we get 
the target in (4.58) as Y(gl n ) -module. Here we used the definition of the Y(gl n ) -module 
^ t ~ fc , and the fact that (4.12) defines an anti-automorphism of the coalgebra Y(g[ n ). 



5. Irreducible representations of Yangians 

5.1. Irreducible representations of Y(gl n ) 

Let ^ be a non-zero finite-dimensional Y(gl n ) -module. A non-zero vector of $ is called 
highest if it is annihilated by all the coefficients of the series Tjj (x) with i < j . If is 
irreducible then a highest vector cp G $ is unique up to a scalar multiplier. Moreover, 
then (f is an eigenvector for the coefficients of all series Tu(x) , and for i = 1 , . . . , n — 1 



T u (x) T i+ i ji+ i(x) _1 ip = P t (x + \) P^x - i)" 1 



where Pi(x) is a monic polynomial in x with coefficients in C. Then P\{x) , . . . , P n -\{x) 
are called the Drinfeld polynomials of $ . Any sequence of n — 1 monic polynomials with 
complex coefficients arises in this way. Furthermore, two irreducible finite-dimensional 
Y(gl n ) -modules have the same Drinfeld polynomials if and only if their restrictions to 
the subalgebra SY(gl n ) C Y(gl n ) are equivalent. Thus up to equivalence and similarity, 
all the non-zero irreducible finite-dimensional Y(gl n ) -modules are parametrized by their 
Drinfeld polynomials [D2, Theorem 2]. For example, consider the trivial Y(gl n ) -module. 
It is one-dimensional, and is defined by the counit homomorphism Y(g( n ) — > C. Then 
the corresponding Drinfeld polynomials are also trivial: Pi(x) = . . . = P n -i(x) = 1. 

In this subsection we will assume that 6 = — 1 and g = gl m , so that g' = g[ n . For 
any k G {0,1, ...,n} the vector space <E> k is irreducible under the action of gl n . Hence 
for any t G C the Y(gl n ) -module is irreducible. The vector <pk = fi A . . . A fk of this 
module is highest, see the definitions (4.6) and (4.8). Moreover, by these definitions 



Tu(x) Lp k 



(x — t + 1) (x — t) 1 <fk if 1 ^ i ^ k ; 

<fik if k < i ^ n. 



Hence for i = 1 , . . . , n — 1 the Drinfeld polynomial Pi(x) of the Y(g( n ) -module ^ is 

Pi(x) = 



x — t + ^ if i = k] 
1 otherwise. 
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For g = g( m we have v = A — p, . Suppose that each label v a e {0 , 1 , . . . , n} . Further 
suppose that the weight A + p is nonsingular. The latter condition means here that 

A& — A a + p\, — p a 7^ 1, 2, . . . for all 1 ^ a < b ^ m. (5.1) 

By Corollary 3.9 and Proposition 4.1, the quotient by the kernel of our operator (4.49) 
is then an irreducible Y(g[ n ) -module. But the definition (4.10) of the comultiplication 
on Y(g( n ) implies that the vector (p Ul <g> . . . <g> (p Vm of (4.46) is highest. Suppose that 

v a ^ Uf, whenever A a + p a = A& + Pb and a < b. (5.2) 

Due to the nonsingularity of the weight A + p, then Proposition 4.4 implies that the 
image of the vector (p Ul ® . . . <g) ip Vm in the quotient is not zero. Hence this image is 
highest relative to the action of the Yangian Y(gl n ) on the quotient. Since the vectors 
cp Ul , . . . , (p Vm are highest in their Y(gl n ) -modules, their tensor product is an eigenvector 
for all coefficients of the series Tu{x) , see again (4.10). Moreover, 

Tu(x) (cp Ul (8) ... (8) (p Vm ) = (Ta(x) ip Vl ) <g> . . . <g> (Tu(x) ip Um ) . 

Therefore for any index i = 1 , . . . , n — 1 the Drinfeld polynomial Pi(x) of the quotient is 
equal to the product of the Drinfeld polynomials with the same index i of the m tensor 
factors of (4.46). Thus we get the following theorem. Recall that here p a = m/2 — a + ^ . 

Theorem 5.1. Let the labels Ai , . . . , A m satisfy the condition (5.1), while the labels 

V\ = Al — p,\ , . . . , V m = A m — p m 

belong to the set {0, 1 , . . . , n} and satisfy the condition (5.2). Then the quotient by the 
kernel of our intertwining operator (4.49) is a non-zero irreducible Y(gl n ) -module. For 
any % = 1 , . . . , n — 1 the Drinfeld polynomial P{(x) of this module is the product of the 
differences x — p a — p a taken over all indices a such that v a = % . 

Note that if v\ = . . . = v m = 0, then both the source and the target Y(gl n ) -modules 
in (4.49) are trivial for any p. If moreover A + p is nonsingular, then by Proposition 4.4 
our operator (4.49) is the identity map C — > C. 

Now take any sequence of n — 1 monic polynomials P\(x) , . . . , P n -i(x) with complex 
coefficients. Denote by V the collection of pairs (i,z) where i = 1 , . . . , n — 1 and z ranges 
over all roots of the polynomial Pi(x) ; the roots are taken with their multiplicities. Note 
that the collection V is unordered. Let m be the total number of elements in V , it is 
equal to the sum of the degrees of Pi(x) , . . . , P n -\(x) . Suppose that at least one of the 
polynomials is not trivial, so that m > 0. Let A and p, be any weights of gl m such that 
for v = A — p the collection of pairs ( v a , p, a + p a ) for a = 1 , . . . , m coincides with V . In 
particular v a e {1, . . . ,n— 1} . We do not yet impose any other conditions on A and p. 

Then A and p are determined up to any permutation of the m pairs [v a , p a + p a ) . 
Equivalenly, they are determined up to a permutation of the m pairs ( A a + p a , p a + p a ) . 
In other words, A and p are determined up to the (simultaneous) shifted action of the 
Weyl group & of gl m on them, as on elements of t*. Therefore we can choose A to satisfy 
the conditions (5.1), so that the weight A + p is nonsingular. For a < b the equality on 
the right hand side of (5.2) means that A is invariant under the shifted action of that 
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element of & which exchanges n a with 775 , and leaves all other basis vectors of t* fixed. 
This action amounts to exchanging the pair ( v a , p a + p a ) with ( z/5 , pb + pb) ■ By using 
this action and keeping A fixed, we can choose \i so that the condition (5.2) is satisfied. 

Now consider our operator (4.49) corresponding to the weights A and \x chosen above. 
Due to Theorem 5.1, the quotient by the kernel of this operator is a non-zero irreducible 
Y(gl n ) -module, and has the given Drinfeld polynomials P\(x) , . . . , P n -i(x) . Thus up to 
equivalence and similarity, every non-zero irreducible finite-dimensional Y(gl n ) -module 
arises as such a quotient. Note that the choice 6 = — 1 here is essential. 

5.2. Representations of twisted Yangians 

From now on we will regard the coefficients of all the series Sij(x) as generators of the 
algebra Y(rj') , with the relations (4.14) and (4.18) imposed on them. In this subsection 
we collect some general results on finite-dimensional Y(q') -modules for both q' = sp n 
and q' = so n . Let & be any of these modules. If ^ is obtained by restricting the trivial 
Y(fjl n ) -module to the subalgebra Y(rj') C Y(rjt n ), then W will be also called trivial. 

Now assume that ^ is non-zero. We will use the ordering (4.56) of the indices 1 , . . . , n . 
We will keep writing i ~< j when i precedes j in the sequence (4.56). A non-zero vector 
of & is called highest if it is annihilated by all the coefficients of the series Sij(x) with 
i -< j . If \P is irreducible then a highest vector tfj G \P is unique up to a scalar multiplier. 
Then ift is an eigenvector for the coefficients of all series Su(x) ; see [M2, Theorem 4.2.6 
and Corollary 4.2.7]. Then & is determined by the corresponding eigenvalues up to 
equivalence. In the next subsections we will use the description from [M2, Chapter 4] of 
all possible eigenvalues. Note that the relation (4.18) with i = j takes the form 



Hence it suffices to describe the eigenvalues of the coefficients of only one of every two 
series Sa(x) and Sii(x). We choose the series Su(x) with i )p 1, or equivalently with 
i )p n. This choice is explained by the next lemma which valid for any non-zero Y(gl n )- 
module ^, not necessarily irreducible; cf. [M2, Corollary 4.2.10]. We will also regard ^ 
as module over the subalgebra Y(g') C Y(gl n ) , by restriction. 

Lemma 5.2. Let <p G $ be a non-zero vector annihilated by all coefficients of the series 
Tij(x) with i < j . Suppose Tu(x) <p = hi(x) <p for i = 1 , . . . , n where hi(x) is a formal 
power series in x~ x with coefficients from C. Then <p is a highest vector for Y(rj') and 
moreover Sa (x) ip = gi(x) ip for i = 1 , . . . ,n where 



Proof. Take i ,j G {l,...,n} such that i =^ j . The coefficients of the scries S^jijx^ cict on 
the vector <p G ^ as the corresponding coefficients of the series (4.13), giving the sum of 



over the indices k ^ j . This is because the vector cp is highest relative to Y(rj[ n ) . Using 
the commutation relations (4.5), the summand (5.3) is equal to the difference 



Su(x) =f 2xSu{x) = (1t2x) Su(-x). 




9 t e k T- kl (-x)T kj (x)^ 



(5.3) 
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e i e k T kj (x)T~ k -(-x)<f (5.4) 

- di6 k (2x)- 1 (T kl (-x) T- kj (x) - T kl {-x) T~ kj {x)) p. (5.5) 

Here (5.4) may be non-zero only if k )p i, that is only if k =4 i- The latter condition 
together with i ^ j and j =4 k implies that i = j = k. Then (5.4) equals 

hi(x) hi(-x) p. (5.6) 

Further, (5.5) may be non-zero only if k y- j . When presenting (5.3) as the sum of 
(5.4) and of (5.5) by using (4.5), we could replace (5.5) by the expression 

- 6i6 k (2x)- 1 (T- kj (x) T fcf (-x) - T kj (x) T kl {-x)) <p. (5.7) 

But (5.7) may be non-zero only if k y- i, that is only if A; =4 i. The latter condition 
together with i j and j ^ k implies that i = j = k. Then (5.5) and (5.7) are equal to 

T(2x)~ 1 (hi(x) h % {-x) - hi(-x) h % {x)) p. (5.8) 

We have also assumed that k y- j , which together with the equalities i = j = k implies 
that i =^ ?. Moreover, if i = % then (5.8) obviously vanishes. 

Thus under the assumption % ^ j we have proved that Sij (x) <p may difer from zero 
only of i = j . If i -< i then Su(x) ip is equal to the sum of (5.6) and (5.8). But if % )p l 
then Sa(x) ip is equal to the expression (5.6) alone. □ 

We will keep assuming that = — 1. Hence for g = sp 2m or g = S02 m we will have 
q' = sp n or q' = so n respectively. In these next subsections, we will consider the two 
cases separately. Moreover, we will separate the cases of fj' = so n with n even and odd. 
If q' = sp n then n has to be even. We will write n = 21 if n is even, or n = 21 + 1 if n 
is odd. Then the condition i n will mean respectively that i is one of the indices 

2/,2/-2,...,2 or 21 + 1 , 21 , 21 - 2 , . . . , 2 . 

As the first application of Lemma 5.2, consider the case when ^ = (f>\ and p = tfj k is 
the vector of <I? k defined in Subsection 4.4. We assume that k G {0,1, . . . , n} . All the 
conditions of Lemma 5.2 are then satisfied. Here hi{x) = 1 + (x — t) -1 if i is one of the 
first k indices in the sequence (4.56), otherwise hi(x) = 1. Hence hi(—x) = 1 — (x + t) -1 
if % is one of the last k indices in the sequence (4.56), otherwise h%{— x) = 1. If i )p n, 
then gi(x) = hi(x) h^(—x). This implies the following relations, to be used later on. 

Take any i y n. Let j be the index occuring just before i in the sequence (4.56). Here 
j = i + 2 unless n = 21 + 1 and i = 21 , in which case j = i + 1 . Note that j )p n. Then 

{{x + 1) (x + t - l)" 1 if 2k = i; 
(x-t + ^ix-t)- 1 if 2(n-k) = i; 
1 otherwise. 

Indeed, for % y n we have hi(x) ^ hj(x) only if 2{n — k) = i. In the latter case we have 
hi(x) = 1 while hj(x) = 1 + (x — £) -1 . Further, for any i y n we have h%{x) ^ hj(x) 
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only if 2k = i. In the latter case we have h%{x) = 1 + (x — while hj(x) = 1. Thus 
we obtain the relation displayed above. 
If n = 21, then we also have the relation 



g n {-x) g n (x) 



-l 



(x-t + l)(x + t)(x-t)- 1 (x + t-l)- 1 if k = l; 

1 otherwise. 



Indeed, if n = 21 then h = 21 — 1 . Then for k < / we have h n (x) = 1 , while for k > I we 
have h n (x) = 1 + (x — 1)~ . Further, for k < I we have hn(x) = 1, while for k ^ / we 
have hn(x) = 1 + (x — . Therefore g n (x) = g n (~x) for any k ^ I. But if k = I then 
gn( x ) = 1 + (x — t) -1 . Thus we obtain the last displayed relation. 

We will end this section by introducing a notion applicable to any finite-dimensional 
module & over the twisted Yangian Y(rj') where g' = so n and n is any positive integer, 
even or odd. Consider the restriction of & to the subalgebra U(50 n ) C Y(so n ) . Suppose 
that the restriction integrates to a module of the complex special orthogonal group SO n . 
Thus we exclude the spinor representations of so n . Then we call the Y(so n ) -module 
\P integrable. Equivalently, then \P is an (Y(so n ) , SO n ) -module; see the beginning of 
Subsection 4.4. For instance, any Y(so n ) -module of the form (4.51) is integrable, and so 
is any quotient of such a module. We will not treat non-integrable Y(so n ) -modules here. 

5.3. Irreducible representations of Y(sp n ) 

In this subsection, we consider the case when $ = — 1 and g = sp 2m , so that g r = sp n 
where n = 21. Let ^ be a non-zero irreducible finite-dimensional Y(sp n ) -module. Let 
i/j G 1? be a highest vector. By [M2, Theorem 4.3.8] for any k = 1, ...,/ — 1 we have 

S 2 k+2,2k+2(x) S 2 k,2k(x)- 1 ip = Q k (x + \) Qk(x - ip 

where Qk(x) is a monic polynomial in x with coefficients in C. Further, we have 

S nn (-x)S nn (x)- 1 4> = Qi{x + \) Qi{x - ip 

where Qi(x) is an even monic polynomial in x with coefficients in C. Any sequence 
of / monic polynomials with complex coefficients arises in this way, provided that the 
last polynomial in the sequence is even. Furthermore, two irreducible finite-dimensional 
Y(sp n ) -modules have the same sequence of polynomials Qi(x) , . . . , Qi(x) if and only if 
their restrictions to the subalgebra SY(sp n ) C Y(sp n ) are equivalent. Thus the non-zero 
irreducible finite-dimensional Y(sp n ) -modules are parametrized by their polynomials 
Qi(x) , . . . , Qi(x) up to equivalence and similarity [M2, Corollary 4.3.11]. For example, 
if is the trivial Y(sp n ) -module then Qi(x) = . . . = Qi(x) = 1. 

In this subsection, p a = —a and K a = — / for each index a = 1 , . . . , m . Recall the 
definition (2.18) of the weight v. Suppose that each label v a e {0,1,..., n) . Further 
suppose that the weight A + p is nonsingular. The latter condition means here that 

A& — A a + ph — p a 7^ 1, 2, . . . for all 1 ^ a < b ^ m; (5-9) 
A a + Xb + Pa + Pb ^ 1,2, ... for all l^a<b^m; (5.10) 
Ao + Pa ^ 1,2, ... for all l^a^m. (5.11) 
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By Corollary 3.9 and Proposition 4.2, the quotient by the kernel of our intertwining 
operator (4.58) is then an irreducible Y(sp n ) -module. But the definition (4.10) of the 
comultiplication on Y(gl n ) implies that the vector (4.57) is highest relative to the twisted 
Yangian Y(sp n ) . Here we also use Lemma 5.2. Further suppose that 



Due to the nonsingularity of the weight A + p, Proposition 4.6 then implies that the 
image of the vector (4.57) in the quotient is not zero. Hence this image is highest relative 
to the action of the twisted Yangian Y(sp n ) on the quotient. Lemma 5.2 implies that 
each of the polynomials Qi(x) , . . . , Qi(x) of the quotient is multiplicative with respect 
to the m tensor factors of the vector (4.57). But in the case m = 1 these polynomials 
are transparent from the relations given in the very end of Subsection 5.2. Thus we get 

Theorem 5.3. Put n = 21. Let Ai , . . . , A m satisfy (5.9) ; (5.10),(5.11) while the labels 



belong to the set {0,1,..., n} and satisfy (5. 12), (5. 13), (5. 14). Then the quotient by the 
kernel of our intertwining operator (4.58) is a non-zero irreducible Y(sp n ) -module. For 
any k = 1 ,...,/ — 1 the polynomial Qu{x) of this module is the product of the sums 
x + p a + Pa taken over all indices a such that v a = k , and of the differences x — p a — p a 
taken over all indices a such that v a = n — k . The polynomial Qi(x) of this module is 
the product of the differences x 2 — (p a + p a ) 2 taken over all indices a such that v a = I . 

Note that if v\ = . . . = v m = 0, then both the source and the target Y(sp n ) -modules 
in (4.58) are trivial for any p. If moreover A + p is nonsingular, then by Proposition 4.6 
our operator (4.58) is the identity map C — > C. 

Now let us take any sequence of / monic polynomials Qi(x) , . . . , Qi(x) with complex 
coefficients, such that the polynomial Qi(x) is even. Let m be the sum of the degrees of 
Qi(x) , . . . , Qi-i(x) plus half of the degree of Qi(x) . Suppose that at least one of all the 
/ polynomials is not trivial, so that m > 0. Let A and p be any weights of sp 2m such 
that the corresponding labels (5.15) belong to the set {1 , . . . , n — 1} and such that the 
given polynomials Q\{x) , . . . , Qi(x) are obtained from A and p as in Theorem 5.3. But 
we do not yet impose any other conditions on A and p , such as nonsingularity of A + p . 

Then A and p are determined up to permuting the m pairs ( z/ a , p a + p a ) , and up to 
replacing (u a , p a + Pa) by (n — v a ,—p a — p a ) for any number of indices a . Equivalenly, 
A and p are determined up to a permutation of the m pairs ( A a + p a , p a + Pa ) , and 
up to replacing ( A a + p a , p a + Pa ) by (-A a - p a , -p a - p a ) for any number of indices 
a. In other words, A and p are determined up to the (simultaneous) shifted action of 
the Weyl group & of sp 2m on them, as on elements of t*. Therefore we can choose A to 
satisfy the conditions (5. 9), (5. 10), (5. 11) so that the weight A + p is nonsingular. 

The equality on the right hand side of (5.12) for a < b means that A is invariant 
under the shifted action of that element of & which exchanges ?? m _ a+ i with ?7 m _b+i , 
and leaves all other basis vectors of t* fixed. This action amounts to exchanging the 



v a ^ Vb whenever A a — A;, + p a — Pb = and a < b; 
v a + ^b ^ n whenever A a + A;, + p a + Pb = and a < b; 
u a ^ / whenever A a + p a = . 



(5.12) 
(5.13) 
(5.14) 



V\ — Ai — p\ + I , . . . , V m — A m — p m + I 



(5.15) 
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pair (v a , p a + p a ) with ( i/& , pb + p ) . By using this action and keeping A fixed, we can 
choose p so that the condition (5.12) is satisfied. 

The equality on the right hand side of (5.14) means that the weight A is invariant 
under the shifted action of that element of & which maps rj m - a+ i to — r/ m _ a +i, and 
leaves all other basis vectors of t* fixed. This action amounts to replacing the pair 
( v a , p, a + p a ) by the pair ( n — v a , — p a — p a ) . By using this action and keeping A fixed, 
we can choose p so that the condition (5.14) is satisfied. 

Finally, the equality on the right hand side of (5.13) for a < b means that the weight A 
is invariant under the shifted action of that element of the group & which maps rj m - a+ i 
and rj m -b+i respectively to — n m -b+i and — r] m - a+ i, leaving all other basis vectors of 
t* fixed. This action amounts to replacing the pairs (v a ,p a -\- pa) and (vt, , pb + p ) by 
the pairs (n — Ub,—pb — Pb) and (n — v a ,—p a — p a ) respectively. By using this action 
and keeping A fixed, we can choose the weight p so that the condition (5.13) is satisfied 
when A a + p a ^ , or equivalently when Xb + pb ^ . When A a + p a = and Xb + Pb = , 
the condition (5.13) is already satisfied, because then v a ^ / and r\b ^ / due to (5.14). 

Now consider our operator (4.58) corresponding to A and p. Due to Theorem 5.3 the 
quotient by the kernel of this operator is a non-zero irreducible Y(sp n ) -module, and has 
the given polynomials Q±(x) , . . . , Qi(x) . Thus up to equivalence and similarity, every 
non-zero irreducible finite-dimensional Y(sp n ) -module arises as such a quotient. 

5.4- Irreducible representations of Y(so n ) for odd n 

In this subsection, we consider the case of g' = so n where n is odd. Hence n = 21 + 1 
where / is a non-negative integer. We assume that 9 = — 1 , so that g = S02 m • Let ^ be 
any integrable finite-dimensional Y(so n ) -module, see the end of Subsection 5.2. Further 
suppose that the Y(so n ) -module \P is irreducible and non-zero. Let ip G \P be a highest 
vector. By [M2, Theorem 4.5.9] there exist monic polynomials Q±(x) , . . . , Qi(x) in x 
with complex coefficients such that for any k = 1 ,...,/ — 1 

S2k+2,2k+2(x) S 2 k,2k(x)~ 1 1p = Qk(x + \) Qk{x - |) _1 ^, 

while 

S nn (x) Sn-l,™-!^)" 1 if) = Ql(x + \) Ql(x - i)" 1 lp . 

Every sequence of / monic polynomials with coefficients from C arises in this way. Two 
irreducible integrable finite-dimensional Y(so n ) -modules have the same polynomials 
Qi(x) , . . . , Qi(x) if and only if their restrictions to the subalgebra SY(so n ) C Y(so n ) 
are equivalent [M2, Corollary 4.5.12]. Thus all the non-zero irreducible integrable finite- 
dimensional Y(so n ) -modules are parametrized by their polynomials Qi(x) , . . . , Qi(x) 
up to equivalence and similarity. For example, if ^ is the trivial Y(so n ) -module then 
we have Qi(x) = . . . = Qi(x) = 1 . 

In this subsection, p a = 1 — a and K a = — / + \ for each index a = 1 , . . . , m . Recall 
the definition (2.18) of the weight v . Suppose that each label v a 6 {0 , 1 , . . . , n} . Further 
suppose that the weight A + p is nonsingular. The latter condition means here that 

A& — A a + pb — p a 7^ 1) 2, . . . for all 1 ^ a < b ^ m; (5.16) 
K + h + Pa + Pb 7^ 1,2, ... for all 1 ^ a < b ^ m. (5.17) 
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In the end of Subsection 4.4 we observed that for g = so 2m our intertwining operators 
(4.58) and (4.59) are the same. Using Proposition 4.2 and Corollary 4.9, the quotient by 
the kernel of this operator is an irreducible Y(so n ) -module. But the definition (4.10) of 
the comultiplication on Y(rjt n ) implies that the vector (4.57) is highest relative to the 
twisted Yangian Y(so n ) . Here we also use Lemma 5.2. Further suppose that 

v a ^ whenever A a — A& + p a — pb = and a < b; (5.18) 
v a + v b ^ n whenever A a + A& + p a + pb = and a < b. (5.19) 

We have already noted that for a = o"o and u = T m o~o our operator (4.53) is invertible. 
Due to the nonsingularity of the weight A+p, Proposition 4.6 now implies that the image 
of the vector (4.57) in the quotient is not zero. Hence this image is highest relative to 
the action of the twisted Yangian Y(so n ) on the quotient. Lemma 5.2 implies that each 
of the polynomials Qi(x) , . . . , Qi(x) of the quotient is multiplicative with respect to 
the m tensor factors of the vector (4.57). But in the case m = 1 these polynomials are 
transparent from the relations given in the very end of Subsection 5.2. Thus we get 

Theorem 5.4. Put n = 21 + 1 . Let Ai , . . . , A m satisfy (5. 16), (5. 17) while the labels 

v\ = Ai - lit + / + \ , . . . , v m = \ m - p, m + / + \ (5.20) 

belong to the set {0,1,..., n} and satisfy (5. 18), (5. 19). Then the quotient by the kernel 
of our intertwining operator (4.58) is a non-zero irreducible Y(so n ) -module. For any 
k = 1 , . . . , I the polynomial Qk(x) of this module is the product of the sums x + p a + p a 
taken over all indices a such that v a = k , and of the differences x — p a — p a taken over 
all indices a such that v a = n — k . 

Note that if v\ = . . . = v m = 0, then both the source and the target Y(so n ) -modules 
in (4.58) are trivial for any p. If moreover A + p is nonsingular, then by Proposition 4.6 
our operator (4.58) is the identity map C — > C. 

Now let us take any sequence of / monic polynomials Qi(x) , . . . , Qi(x) with complex 
coefficients. Let m be the sum of the degrees of Qi(x) , . . . , Qi(x) . Suppose that m > 0. 
Let A and p be any weights of S02 m such that the corresponding labels (5.20) belong to 
the set {1 , ... , n — 1} and such that the given polynomials Qi(x) , . . . , Qi(x) are obtained 
from A and p as in Theorem 5.4. We do not yet impose any other conditions on A and p. 

Then A and p, are determined up to permuting the m pairs ( v a , p a + p a ) , and up to 
replacing (v a , p a + p a ) by {n — v ai —p a — p a ) for any number of indices a . Equivalenly, A 
and p are determined up to a permutation of the m pairs ( A a + p a , p a + Pa ) , and up to 
replacing ( A a + p a , p a + p a ) by (-A a - p a , -p a - p a ) for any number of indices a . Thus 
A and p, are determined up to the (simultaneous) shifted action of the group 91 of S02 m 
on them, as on elements of t*. Using only the action of the subgroup & C £H, we can 
choose A to satisfy the conditions (5. 16), (5. 17) so that the weight A + p is nonsingular. 

The equality on the right hand side of (5.18) for a < b means that A is invariant 
under the shifted action of that element of & which exchanges rj m - a+ i with i] m -b+i , 
and leaves all other basis vectors of t* fixed. This action amounts to exchanging the pair 
{.Va^pa^ p a ) with (uf,, Pb + Pb) ■ By using this action and keeping A fixed, we can choose 
p so that the condition (5.18) is satisfied. 



Mickelsson algebras and representations of Yangians 73 

The equality on the right hand side of (5.19) for a < b means that the weight A is 
invariant under the shifted action of that element of the group & which maps r) m - a +i 
and rjm-b+i respectively to — rfm-b+i and — r] m - a +i, leaving all other basis vectors of 
t* fixed. This action amounts to replacing the pairs (v a , fi a + p a ) and (vb , fib + Pb) by 
the pairs (n — i>b,—fib — Pb) and (n — v a ,—fi a — p a ) respectively. By using this action 
and keeping A fixed, we can choose the weight fi so that the condition (5.19) is satisfied. 

Now consider our intertwining operator (4.58) corresponding to these A and fi. Due 
to Theorem 5.4, the quotient by the kernel of this operator is a non-zero irreducible 
integrable Y(so n ) -module, and has the given polynomials Qi(x) , . . . , Qi(x) . Thus up 
to equivalence and similarity, every non-zero irreducible integrable finite-dimensional 
Y(so n ) -module arises as such a quotient. Note that the choice 9 = — 1 here is essential. 

5.5. Irreducible representations of Y(so n ) for even n 

In this subsection, we will consider the case of q' = so n where n is even. Hence n = 21 
where / is a positive integer. We keep assuming that 9 = — 1, so that g = S02m- Let \P 
be any Y(so n ) -module. Let v G O n be the element which exchanges the basis vector 
f n -i with f ni and leaves all other basis vectors of C n fixed. Consider the corresponding 
automorphism (4.52) of the algebra Y(so n ) . Our element v is involutive, and so is the 
corresponding automorphism of Y(so n ) . Denote by ^ the Y(so n ) -module obtained by 
pulling the action of Y(so n ) on \P back through this automorphism. Any Y(so n ) -module 
equivalent to \P will be called conjugate to \P . 

The Y(so n ) -modules \P and & may be equivalent or not. Suppose that the Y(so n )- 
module ^ is irreducible, finite-dimensional, integrable and non-zero. So is the Y(so n )- 
module ^ then. By [M2, Theorem 4.4.14] for a highest vector ift of at least one of the 
two modules \P and \P we have for k = 1 ,...,/ — 1 

S2k+2,2k+2(x) S 2 k,2k(x)~ 1 = Q k (x + \) Q k (x - tj)" 1 (5.21) 

where Qk(x) is a monic polynomial in x with coefficients in C. Further, we have 

S nn (-x)S nn (x)- 1 iP = Q^x + DQ^x-l)- 1 ^ (5.22) 

where Qi(x) is an even monic polynomial in x with coefficients in C. Any sequence of 
/ monic polynomials with complex coefficients arises in this way, provided that the last 
polynomial in the sequence is even. Moreover, the two Y(so n ) -modules and & are 
equivalent, if and only if zero is not a root of the corresponding polynomial Qi(x) . 

Let us consider the case when and \P are not equivalent, so that zero is a root of 
the polynomial Qi (x) . Let h be the positive integer such that , 1 , . . . , h — 1 are roots 
of Qi(x) , but h is not. The above vector ip has been a highest vector of one of the two 
Y(so n ) -modules & and & . Let ift be a highest vector of the other of the two. For each 
index i ^ n we have Sa(x) if) = gi{x) ip where gi(x) is a formal power series in x~ l with 
the coefficients from C. The proof of [M2, Theorem 4.4.14] demonstrates that then for 
each i y n we also have the equality Sa(x) ip = gi(x) ip , while 

S nn (x)ijj = g(x) g n (x)$ 

where 
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g(x) = (x + h+\)(x-h+\) 
By the definition of the polynomial Qi(x) here we have 

g n {-x) g n {x)~ l =Qi(x + \) Qi(x - 



1 




However, 




for any polynomial Q(x) , because the integer h is positive. This implies that only one 
of the two non-equivalent Y(so n ) -modules \P and \P gives rise to / polynomials by using 
its highest vector i/j , as in (5.21) and (5.22). If i/j is a highest vector of & , assign to ^ a 
label 6 = 1. Otherwise, that is if ip is a highest vector of \P , assign to \P a label 5 = — 1 . 
The polynomials Qi(x) , . . . , Qi(x) will be from now on associated to both ^ and W . 

Thus to every non-zero irreducible integrable finite-dimensional Y(so n ) -module \P 
we have associated a sequence of monic polynomials Qi(x) , . . . , Qi(x) where the last 
polynomial is even. If zero is a root of the polynomial Qi(x) , that is if & is not equivalent 
to \P , then we also have associated to a label 5 G {+1,-1}. The modules \P are 
parametrized by their polynomials Qi(x) , . . . , Qi(x) and by their labels 5 (where the 
latter exist) up to equivalence and similarity [M2, Corollary 4.4.17]. For example, if ^ 
is the trivial Y(so n ) -module then Qi(x) = . . . = Qi(x) = 1, and there is no label 6 . 

In this subsection q = S02 m , as in Subsection 5.4. In particular, here p a = 1 — a for 
each index a = 1 , . . . , m. Suppose that the weight A + p is nonsingular. This condition 
can be written as the collection of inequalities (5.16) and (5.17). However, now K a = — I. 
Using the definition (2.18) of the weight z/, suppose that each label v a G {0, 1 , . . . , n} . 

In the end of Subsection 4.4 we observed that for q = S0 2 m our intertwining operators 
(4.58) and (4.59) are the same. Denote by N the quotient by the kernel of this operator. 
By Proposition 4.2 and Corollary 4.9, N is either an irreducible Y(so n ) -module or splits 
into a direct sum of two irreducible non-equivalent Y(so n ) -modules. In the latter case, 
our N is irreducible as a module over the algebra O n k Y(so n ) by Proposition 4.8. 

Due to the O n -equivariance of the Olshanski homomorphism X(so n ) — > A 2m , for 
any v G O n pulling the action of Y(so n ) on iV back through the automorphism (4.52) 
amounts to pulling that action forward through the automorphism of X K v X v~ x of 
the algebra End iV . Let us apply this observation to the element v used in the beginning 
of this subsection. If the Y(so n ) -module iV is irreducible, then it is of the form & , where 
^ and & are equivalent. If iV splits to a direct sum of two irreducible non-equivalent 
submodules and one of them is denoted by the other submodule is equivalent to & . 

Like in Subsection 5.4, the definition (4.10) of the comultiplication on Y(gl n ) implies 
that the vector (4.57) is highest for the twisted Yangian Y(50 n ) . Here we again used 
Lemma 5.2. Suppose that the weight v satisfies the inequalities (5.18) and (5.19). Note 
that the weights k used here and in Subsection 5.4 to determine v are different. It is 
the form of the inequalities (5.18) and (5.19) for v that is the same. 

We have already noted that for a = o"o and u> = r m oo our operator (4.53) is invertible. 
Due to the nonsingularity of the weight A+p, Proposition 4.6 now implies that the image 
of the vector (4.57) in the quotient iV is not zero. This image is going to be our vector ift . 
In particular, it will satisfy (5.21) and (5.22) for certain polynomials Qi(x) , . . . , Qi(x) . 
Lemma 5.2 implies that each of these polynomials is multiplicative with respect to the 
m tensor factors of the vector (4.57). But in the case m = 1 these polynomials are 
transparent from the relations given in the very end of Subsection 5.2. Thus we get 



Mickelsson algebras and representations of Yangians 



75 



Theorem 5.5. Put n = 21. Let \\ , . . . , A m satisfy (5. 16), (5. 17) while the labels 

v\ = Ai - ii\ + / , . . . , v m = \ m - p, m + / (5.23) 

belong to the set {0,1, ... ,n} and satisfy (5. 18), (5. 19). Then the quotient by the kernel 
of our intertwining operator (4.58) is either a non-zero irreducible self-conjugateY (so n ) - 
module, or splits into a direct sum of two non-equivalent non-zero irreducible Y(so n ) - 
modules conjugate to each other. For every k = 1 ,...,/ — 1 polynomial Qk(x) of any of 
the irreducible modules is the product of the sums x + p a + p a taken over all indices a 
such that v a = k , and of the differences x — p a — p a taken over all indices a such that 
v a = n — k . The polynomial Qi(x) of any of the irreducible modules is the product of 
the differences x 2 — (p a + p a ) 2 taken over all indices a such that v a = I . The splitting 
occurs if and only if p a + p a = for at least one index a such that v a = I . 

In view of (5.23), the splitting in Theorem 5.5 occurs if and only if A a + p a = and 
Pa + Pa = simultaneously for at least one index a . If v\ = . . . = v m = then both the 
source and the target Y(so n ) -modules in (4.58) are trivial for any p. If moreover A + p 
is nonsingular, then by Proposition 4.6 our operator (4.58) is the identity map C — > C. 

Now let us take any sequence of / monic polynomials Q\{x) , . . . , Qi(x) with complex 
coefficients, such that the polynomial Qi(x) is even. Let m be the sum of the degrees 
of Q\{x) , . . . , Qi-i(x) plus half of the degree of Qi(x) . Suppose that m > 0. Let A and 
p be any weights of S02 m such that the corresponding labels (5.23) belong to the set 
{1 , . . . , n — 1} and such that the given polynomials Qi(x) , . . . , Qi(x) are obtained from 
A and p as in Theorem 5.5. We do not yet impose any other conditions on A and p. 

Then A and p, are determined up to permuting the m pairs [v a , p a + p a ) , and up to 
replacing (u a , p a + p a ) by (n — u a ,—p a — p a ) for any number of indices a . Equivalenly, A 
and p are determined up to a permutation of the m pairs ( A a + p a , p a + p a ) , and up to 
replacing ( A a + p a , p a + p a ) by (— A a — p a , — p a — p a ) for any number of indices a . Thus 
A and p are determined up to the (simultaneous) shifted action of the group Dl of S02 m 
on them, as on elements of t*. Using only the action of the subgroup 6 C SH, we can 
choose A to satisfy the conditions (5. 16), (5. 17) so that the weight A + p is nonsingular. 

By using only the shifted action on p of those elements of the subgroup & C £H which 
leave A invariant, we can choose p so that the conditions (5. 18), (5. 19) are satisfied. The 
arguments are the same as in the end of Subsection 5.4, and we do not repeat them here. 

Now consider our intertwining operator (4.58) corresponding to these A and p. Due to 
Theorem 5.5, the quotient by the kernel of this operator is either a non-zero irreducible 
integrable Y(so n ) -module, or splits into a direct sum of two non-equivalent irreducible 
integrable Y(so n ) -modules. To any of the irreducible modules we associate the given 
polynomials Qi(x) , . . . , Qi(x) . It the quotient is irreducible, there is no label 5 associated 
to it. If the quotient splits into two irreducible modules, they have the labels 5 = 1 and 
5 = — 1 associated to them. Therefore up to equivalence and similarity, every non-zero 
irreducible integrable finite-dimensional Y(so n ) -module arises either as such a quotient, 
or as one of its two direct summands. Note that the choice 9 = — 1 here is essential. 
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